-- Logs begin at Thu 2022-09-15 18:35:39 UTC, end at Thu 2022-09-15 19:45:22 UTC. -- Sep 15 19:06:41.874899 dv-cinder-849846-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 19:06:41.882820 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 19:06:41 2022] *** Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: nodename: dv-cinder-849846-10-iscsi Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: machine: x86_64 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: clock source: unix Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: pcre jit disabled Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: detected number of CPU cores: 4 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: current working directory: / Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: detected binary path: /usr/bin/uwsgi-core Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: your processes number limit is 47680 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: your memory page size is 4096 bytes Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: detected max file descriptor number: 2048 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: lock engine: pthread robust mutexes Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: thunder lock: enabled Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 19:06:41.888145 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 19:06:41.926468 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: Python main interpreter initialized at 0x5621827ff9c0 Sep 15 19:06:41.926468 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: python threads support enabled Sep 15 19:06:41.926468 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: your server socket listen backlog is limited to 100 connections Sep 15 19:06:41.926468 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: your mercy for graceful operations on workers is 90 seconds Sep 15 19:06:41.926854 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 19:06:41.927028 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: *** Operational MODE: preforking *** Sep 15 19:06:41.927201 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: *** uWSGI is running in multiple interpreter mode *** Sep 15 19:06:41.927201 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: spawned uWSGI master process (pid: 81544) Sep 15 19:06:41.927408 dv-cinder-849846-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 19:06:41.928351 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: spawned uWSGI worker 1 (pid: 81558, cores: 1) Sep 15 19:06:41.928351 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: spawned uWSGI worker 2 (pid: 81559, cores: 1) Sep 15 19:06:41.928523 dv-cinder-849846-10-iscsi devstack@keystone.service[81544]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 19:06:46.337675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 19:06:46.337675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: from cryptography.utils import int_from_bytes Sep 15 19:06:46.338492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 19:06:46.338492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: from cryptography.utils import int_from_bytes Sep 15 19:06:46.338927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 19:06:46.338927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: from cryptography.utils import int_from_bytes Sep 15 19:06:46.339035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 19:06:46.339035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: from cryptography.utils import int_from_bytes Sep 15 19:06:46.394406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 19:06:46.394574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 19:06:46.394768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] command line args: [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 19:06:46.394863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 19:06:46.394974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 19:06:46.395108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 19:06:46.395213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] command line args: [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 19:06:46.395295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ================================================================================ {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 19:06:46.395432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 19:06:46.395518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] admin_token = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.395616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ================================================================================ {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 19:06:46.395718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] config_dir = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.395882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.395983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] admin_token = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] config_source = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] config_dir = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] debug = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] config_source = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] debug = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.396877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.397912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_config_append = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_config_append = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_dir = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_file = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_dir = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_file = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.398988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.399940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.400926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.401846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] publish_errors = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] publish_errors = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.402961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] standard_threads = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.403961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] standard_threads = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] use_journal = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] use_json = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.404832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] use_stderr = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] use_journal = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] use_syslog = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] use_json = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] use_stderr = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] use_syslog = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.405829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.405944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 19:06:46.406073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.406949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.407055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.407204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.407305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.407993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.408999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.409989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.410965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.411977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.412926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.413945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.414951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.415985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.416124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.416506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.416866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.416984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.417955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.418943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.419942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.420982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.421908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.422921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.423167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.423167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.423943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.423943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.424929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.425944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.426897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.427949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.428904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.429967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.430907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.431602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.437778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.437909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.437988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.438934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.external = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.external = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.password = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.439935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] auth.token = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.password = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] auth.token = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.440983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.441955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.442975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.443962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.444955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.445990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.446943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.447943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.448977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.449938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.450985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.451959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.452985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.453966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.454957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.455981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.456987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.457945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.458892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.459881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.460940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.461848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.462797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.463808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.464965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] role.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.465953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] role.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] role.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] role.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.466916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.467973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.468943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.469658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.470380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.470507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.470640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.470640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.470864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.471901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.472985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.473866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.474948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.475897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.476981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.477944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.478924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.connection = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.479923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.connection = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.480883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.481901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.482897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.483846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81559) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 19:06:46.484028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.484028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.484028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.484028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 19:06:46.484028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81558) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 19:06:46.511530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.511789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.511883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.512019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.512110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.512333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.520788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.521287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.549830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.550046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.551035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.551271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.616862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.617531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.671708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.671940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.672032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.672209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.847370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.847557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.847719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.847802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.847878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.848672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.875870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.876049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.876133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.876362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.876960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.877882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.888678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.889046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.907094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.907230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.907314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.908910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.909020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.909137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.909239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.909339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81558) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.909439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81559) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 19:06:46.926079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.926285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 19:06:46.926619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.926796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.926908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 19:06:46.927632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 19:06:46.927743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.927743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.927875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.927875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 19:06:46.928004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.928090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.928165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.928236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.928336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.928600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.937465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.937670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.937945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.938231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.938315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.938516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.943761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.943761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.945290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.946114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.948188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.948471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.948757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.949334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.949950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.950960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.951060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.951234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.951510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.951809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.974057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.975086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.975541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.975856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.976609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.976901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.980564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.981016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.981119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.981694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.981985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.982274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.987966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.988257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.989344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.989632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.994115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.994416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.994708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.995004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.995290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.995537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:46.995666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.995864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.995948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.996138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.996227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.996511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.996802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.997082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.997358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:46.997692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.011761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.012037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.013180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.013467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.015923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.017381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.019178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.019481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.020646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.020952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.023734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.025155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.026924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.028334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.030477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.030776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.031063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.031345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.031638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.031842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.031931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.032147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.032962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.033372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.033853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.033853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.048722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.049020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.049290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.049617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.050200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.050499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.050775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.051041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.056398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.057945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.058045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.059469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.059564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.059777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.061068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.061352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.062423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.062719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.063004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.063289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.063756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.064130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.064437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.064727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.065021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.065410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.074871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.075157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.075448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.075754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.076135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.078353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.078430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.091967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.093613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.095369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.097226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.098321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.099860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.100136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.101328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.101627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.101729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.102008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.103133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.103421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.103849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.108372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.109817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.110215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.112258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.113309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.113705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.113994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.114275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.115142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.115545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.115862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.116156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.122861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.123152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.123434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.123986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.124285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.124573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.130372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81558) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.130580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.130855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.131136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.131413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.131497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81559) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 19:06:47.131711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.131798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.131992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.132911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.133922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.134004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81558) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.134207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.134475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.134745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.135043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81559) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 19:06:47.168547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621827ff9c0 pid: 81558 (default app) Sep 15 19:06:47.169696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621827ff9c0 pid: 81559 (default app) Sep 15 19:06:47.172055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435d76-9bf7-4af0-aca1-d0760d74d6d4 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:47.172344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435d76-9bf7-4af0-aca1-d0760d74d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:47.172616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435d76-9bf7-4af0-aca1-d0760d74d6d4 None None] PATH_INFO: `/v3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:47.177375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Thu Sep 15 19:06:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:06:55.138588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:55.139093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:55.139172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:55.143499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:06:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:06:55.148710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:55.149121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:55.149269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:55.221703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: INFO dbcounter [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] Registered counter for database keystone Sep 15 19:06:55.241141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG oslo_db.sqlalchemy.engines [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81558) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 19:06:55.247414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writer thread running {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 19:06:55.428857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG passlib.handlers.bcrypt [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81558) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 19:06:55.429245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG passlib.handlers.bcrypt [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81558) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 19:06:55.487376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:06:55.489012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:06:55.624025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:06:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 19:06:56.086224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:06:56.155976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: INFO dbcounter [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] Registered counter for database keystone Sep 15 19:06:56.178863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG oslo_db.sqlalchemy.engines [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81559) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 19:06:56.185413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writer thread running {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 19:06:56.304995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914aa97-e601-45e0-8fd8-4a34d51a8648 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:06:56.309209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:06:56.309843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:56.310124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:56.310396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:56.319094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:06:56.371776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 19:06:56.372120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93fd68-aa7f-47a6-a3f8-0dbb250a0b79 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:06:56.382351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Sep 15 19:06:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:06:56.388460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:06:56.406801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977eff6-70ef-4df5-98c6-51c5b7a53b36 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:06:56.412278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:06:56.413418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:56.413703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:56.413976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] PATH_INFO: `/v3/projects/6238bc86e5b040f1914e478da12747a8` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:56.415589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] RBAC: Authorizing `identity:get_project(project_id=6238bc86e5b040f1914e478da12747a8)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:06:56.468630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 19:06:56.468891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6593b6-66d7-40a1-9d89-70dd8467afd4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:06:56.474170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Thu Sep 15 19:06:56 2022] GET /identity/v3/projects/6238bc86e5b040f1914e478da12747a8 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:06:58.607847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5eb4-824f-4440-8070-2b7675db40b0 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:58.608363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5eb4-824f-4440-8070-2b7675db40b0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:58.608445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5eb4-824f-4440-8070-2b7675db40b0 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:58.613081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:06:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:06:58.618663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:58.619079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:58.619230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:58.639582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:06:58.641144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:06:58.649572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:06:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 19:06:58.918139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5eb4-824f-4440-8070-2b7675db40b0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:06:58.930237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5eb4-824f-4440-8070-2b7675db40b0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:06:58.934400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:06:58.934992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:58.935274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:58.935550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:58.943742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:06:58.945485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a65d7-ef31-4357-b3f8-e8d6c771071b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:06:58.953989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Sep 15 19:06:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:06:58.960019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:06:58.969335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca697-14e4-4254-a68c-1ff985155bc9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:06:58.973773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:06:58.974399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:06:58.974686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:06:58.974966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:06:58.976312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:06:58.978143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e759b-8409-426b-8cba-75424dbccc08 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:06:58.979689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Thu Sep 15 19:06:58 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:01.210256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888a41-b6b1-4089-8aba-29f95137a268 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.210788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888a41-b6b1-4089-8aba-29f95137a268 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.210788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888a41-b6b1-4089-8aba-29f95137a268 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.212268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:01.217787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.218223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.218317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.237482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:01.237822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c862f-f7d5-44a2-b90c-5c5b3e71befd None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.238118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c862f-f7d5-44a2-b90c-5c5b3e71befd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.238401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c862f-f7d5-44a2-b90c-5c5b3e71befd None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.238657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:01.239857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:01.244989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.245280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.245561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.288276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG passlib.handlers.bcrypt [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81559) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 19:07:01.288675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG passlib.handlers.bcrypt [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81559) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 19:07:01.328729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:01.337907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:01.339528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:01.442878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:01.599303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:01.625406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:01.710595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:01.722558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597ced4-3391-4dd9-a916-3c67e03752b6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:01.757140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:01.757781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.758059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.758340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.771543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:01.780732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:01.781795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:01.783509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:01.788239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:01.823406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ce50f-6614-4813-a08f-4442b19ec1fa None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:01.827020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:01.874577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:01.874577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.874577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.874577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.875037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:01.895042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939c2b-01f7-49ac-83a6-5426af33443e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:01.936776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:01.937402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:01.937688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:01.937970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:01.939239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:01.939575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:01.941027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:01.945793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:01.947301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:01.953248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:01.959533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:02.049474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 19:07:02.055352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:02.058876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:02.061229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167ab1b-0440-4da1-814d-6328a08cacb8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:02.096947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:02.097604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:02.097900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:02.098182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:02.099362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:02.101048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:02.102661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:01 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:02.108087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:02.122717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:02.159608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a1d63-cf65-41e6-b533-6e427f384b38 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:02.193375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:02.194237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:02.194521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:02.194802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:02.212469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcc0f-4213-420c-b90a-32701338d40c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:02.213911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:02.215399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:07:02.234957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:02.236156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:07:02.236956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:07:02.236956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:07:02.236956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:02.236956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:02.242117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:02.248650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:02.249295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:02.249576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:02.249850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:02.251534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:02.253018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:02.254590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:02.260346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:02.303636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:02.352251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94ce7-5d65-4ca1-8786-8e1ce49c03ab None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:02.354355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:02.374607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4873-28d5-4a4f-a4ca-ad4809a93c50 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:02.389171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:02.389821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:02.393723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:02.393723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:02.393723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:02.393723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:02.393723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21cc7-7a62-4c38-b06f-190f3f47a26e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:02.409958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:02.410503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:02.410784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:02.410906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:02.411116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:02.412414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.domain.id=default, user.id=ddb604f3212045e99bbc5f99912931b2) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:02.412897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:02.414825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e9e1a-398b-47e9-b443-f5371e4b2285 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:02.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Sep 15 19:07:02 2022] GET /identity/v3/role_assignments?user.id=ddb604f3212045e99bbc5f99912931b2&scope.domain.id=default&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:05.181720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1feb-f3fc-496e-a120-b0bf2f501f4d None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.182218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1feb-f3fc-496e-a120-b0bf2f501f4d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.182218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1feb-f3fc-496e-a120-b0bf2f501f4d None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.183702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:05.189076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d11491-1418-4681-bb6d-d8b062792dfe None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.189518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d11491-1418-4681-bb6d-d8b062792dfe None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.189614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d11491-1418-4681-bb6d-d8b062792dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.193946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85ae7-54d6-4a3b-b2dc-f30c27334d38 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.194232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85ae7-54d6-4a3b-b2dc-f30c27334d38 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.194510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85ae7-54d6-4a3b-b2dc-f30c27334d38 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.195990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 12/23] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:05.205384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79bfe-efbc-41aa-a527-2a4624589f73 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.205681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79bfe-efbc-41aa-a527-2a4624589f73 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.205946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79bfe-efbc-41aa-a527-2a4624589f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.234072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b7d11491-1418-4681-bb6d-d8b062792dfe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:05.240289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b7d11491-1418-4681-bb6d-d8b062792dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:05.243332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0ad79bfe-efbc-41aa-a527-2a4624589f73 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:05.245258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0ad79bfe-efbc-41aa-a527-2a4624589f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:05.388532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 13/24] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:05.389403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:05.533578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a588f-4e87-43b5-8782-878a93a1c84a None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.533915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a588f-4e87-43b5-8782-878a93a1c84a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.534153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a588f-4e87-43b5-8782-878a93a1c84a None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.535511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 14/26] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:05.541875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:05.542399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:05.542878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:05.568151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:05.569384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:05.654811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:05.759192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a588f-4e87-43b5-8782-878a93a1c84a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:05.824923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:05.873798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:05.878197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:05.932487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a588f-4e87-43b5-8782-878a93a1c84a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.033118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.033773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.034064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.034347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.035052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.036563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.064186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19b52f-a003-4e9c-99cc-27ed685f7fe6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.127528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.128142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.128433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.128715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.143220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:06.148711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:06.149564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:06.150443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:06.156626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.274212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:06.283342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c733-6a58-4081-862b-65c2e4bd97ac None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.297402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 15/29] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 15 19:07:05 2022] POST /identity/v3/roles => generated 219 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:06.301033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.361506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.362195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.362482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.362757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.363454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:06.363791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.365268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4cc3c-aa23-4a1d-83dc-5ebb58f93be8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.381391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:06.382580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd731a3-ac87-4206-bc3f-80ba0206935a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.382874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd731a3-ac87-4206-bc3f-80ba0206935a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.383156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd731a3-ac87-4206-bc3f-80ba0206935a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.384412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 16/31] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:06.395244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.395544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.395842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.436760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:06.438013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:06.459480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5d038-fc90-47f4-bc76-4cd8098a4e5e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.474648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:06.502998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.504069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.504377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.504658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.516497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.518649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:07:06.536787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:06.537709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:07:06.538544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:07:06.538544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:07:06.538544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:06.542471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 16/32] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:06.543602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.598024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 17/33] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:06.599469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.661292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd3c9-d05e-4ad8-9e77-7e7a6114859d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.665565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:06.696893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.697534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.697825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.698104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.699919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.701533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d51e0a-190e-4530-a7e6-b1cff8b19bce None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.703152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:06 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:06.704803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-34cf-4d7a-aa3c-5e1c211588e0 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.704934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-34cf-4d7a-aa3c-5e1c211588e0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.705134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-34cf-4d7a-aa3c-5e1c211588e0 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.706602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 18/35] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:06.711383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572365e-34cf-4d7a-aa3c-5e1c211588e0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.718811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143d03f-e866-4aa5-94fb-9665cbbe85b5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.742798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:06.760089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.760932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.761511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.761511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.762139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:06.762453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.764247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f1ee-efae-4676-a05a-b4b22134cec5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.783386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 18/36] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:06.785079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee763b-59e3-4001-b542-9aad43dff6d8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.785370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee763b-59e3-4001-b542-9aad43dff6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.785645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee763b-59e3-4001-b542-9aad43dff6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.804588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ecee763b-59e3-4001-b542-9aad43dff6d8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:06.805715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ecee763b-59e3-4001-b542-9aad43dff6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:06.841141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572365e-34cf-4d7a-aa3c-5e1c211588e0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:06.877165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 19/37] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:06.878423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f73e-092f-48fe-b7a2-fe045ba10e9b None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.878715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f73e-092f-48fe-b7a2-fe045ba10e9b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.878994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f73e-092f-48fe-b7a2-fe045ba10e9b None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.880456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:06.884975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3f73e-092f-48fe-b7a2-fe045ba10e9b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:06.888111 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:06.888751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.889040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.889319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.891194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:06.892786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186f6e-a0b2-4b64-9fb4-5004ef8c98b0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:06.894517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 19/39] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:06.895672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:06.895971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:06.896254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:06.920529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:06.922002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:06.948566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:07.003676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 20/40] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:07.003676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:07.024266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3f73e-092f-48fe-b7a2-fe045ba10e9b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:07.038219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:07.062823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:07.064126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:07.064351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:07.064616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:07.065265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:07.066799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:07.131127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165fc41-6ee9-4778-9330-457487d75fae None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:07.168252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:07.168930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:07.169214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:07.169489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] PATH_INFO: `/v3/domains/default/users/ddb604f3212045e99bbc5f99912931b2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:07.171513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ddb604f3212045e99bbc5f99912931b2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:07.173511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:07.264398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 21/41] 10.222.0.18 () {58 vars in 1361 bytes} [Thu Sep 15 19:07:06 2022] PUT /identity/v3/domains/default/users/ddb604f3212045e99bbc5f99912931b2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:07.265997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:07.304363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 21/42] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:07:06 2022] POST /identity/v3/projects => generated 301 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:07.305437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:07.443166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:07.473852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04234e4-c87e-4373-a177-0e7aaaa2562a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:07.539696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:07.540315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:07.540608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:07.540893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:07.542072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:07.543474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:07.607958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:07.612637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63155d0-e777-4de2-b261-16956edf672e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:07.623633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 22/43] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 15 19:07:07 2022] POST /identity/v3/roles => generated 225 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:07.625797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:07.661105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:07.661749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:07.662697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:07.662997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:07.663742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:07.665703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:07.760154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3e86-9665-4ea8-895c-6441eaa248a8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:07.773797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 22/44] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 15 19:07:07 2022] POST /identity/v3/roles => generated 223 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:07.810894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:07.811568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:07.811991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:07.812278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:07.814804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:07.816503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3681d3-2dae-4193-b5c2-6b329ded555f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:07.818292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 23/45] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:07.824577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:07.963408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4cee-a641-4b35-bdd7-d2b698005e8b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:08.005298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:08.005933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:08.006221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:08.006509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:08.007184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:08.008749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4d507-fd7a-49e4-b769-f127e032fabe None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:08.103866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 23/46] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:07:07 2022] POST /identity/v3/projects => generated 312 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:09.513340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a271ea-e59c-4cca-a21c-8b8ef9125ff2 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:09.513853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a271ea-e59c-4cca-a21c-8b8ef9125ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:09.513936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a271ea-e59c-4cca-a21c-8b8ef9125ff2 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:09.515396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 24/47] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:09.521088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:09.521507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:09.521660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:09.540067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:09.541577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:09.544327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:09.610798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 24/48] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:09.880619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a271ea-e59c-4cca-a21c-8b8ef9125ff2 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:09.895639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:09.983781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a271ea-e59c-4cca-a21c-8b8ef9125ff2 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.018337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.018969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.019259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.019540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.031004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:10.037527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:10.038363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:10.039192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 25/49] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:10.045394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.146640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687c83d-d731-487d-85d3-5969d93337a5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.184717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.185352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.185639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.185919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.186609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:10.186917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.188433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292c39e-fcaa-4c53-85a5-fa7d215059bb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:10.197414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 25/50] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:10.203319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.277745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:10.283190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d04c-90a9-4f16-a492-3c1d2764eb8a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.283482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d04c-90a9-4f16-a492-3c1d2764eb8a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.283803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d04c-90a9-4f16-a492-3c1d2764eb8a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.285240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 26/51] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:10.306435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.306435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.306435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.306435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:10.312169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:10.314004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:10.393592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 27/52] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:10.423211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6804156-9e9c-471b-b3e5-3b20f86e287b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.455864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.456507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.456794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.457082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.458658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.460630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:10.462228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 26/53] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:10 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:10.467999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.582256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613b315-3c54-4116-9241-e1b89affce0a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.612681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:10.618623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:10.625667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 28/54] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:10.630114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.674724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.737701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e335a8a-7b53-4b9e-9e9e-c42c5f9f261d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.774576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.775190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.775558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.775862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.776593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.domain.id=default, user.id=ddb604f3212045e99bbc5f99912931b2) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:10.776912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.778346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:10.790397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 27/55] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Sep 15 19:07:10 2022] GET /identity/v3/role_assignments?user.id=ddb604f3212045e99bbc5f99912931b2&scope.domain.id=default&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:10.794550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a949e8-3a14-4bf6-9ddc-7feac13f3625 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.833410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.834063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.834349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.834625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.835801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.837285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0cd6d-6736-4e14-832f-e12b12211bcf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:10.838936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 29/56] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:10.845480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:10.849602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:10.957547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974791e8-9465-45fd-9b6c-0ed9891661dc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:10.994061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:10.994694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:10.994987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:10.995269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:10.995969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:10.997415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864478eb-369e-40f1-8ec3-8b0a7363e1ae None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:11.092345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 28/57] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Sep 15 19:07:10 2022] POST /identity/v3/projects => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:13.243183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010425a1-429e-4d54-8d33-838551a2f6a3 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:13.243758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010425a1-429e-4d54-8d33-838551a2f6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:13.243758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010425a1-429e-4d54-8d33-838551a2f6a3 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:13.245099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 30/58] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:13.250275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:13.251171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:13.251444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:13.270060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:13.271298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:13.320526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:13.337116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 29/59] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:13.604604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010425a1-429e-4d54-8d33-838551a2f6a3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:13.670830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:13.705141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010425a1-429e-4d54-8d33-838551a2f6a3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:13.739469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:13.740132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:13.740418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:13.740698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:13.741834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:13.743294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30fc39-632d-40be-80b7-9e8ad9d2423e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:13.744914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 31/60] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:13.750638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:13.854215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000403a0-79f1-40b1-9aa4-b1660a32982a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:13.889083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:13.889624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:13.889909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:13.890195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:13.890853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:13.892365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945fc85-5822-4381-b26c-3c75e4278c1d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:14.005664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 30/61] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:13 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:17.445151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece38d-4514-4b4b-bc8d-f029490fbea1 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.445151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece38d-4514-4b4b-bc8d-f029490fbea1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.445151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece38d-4514-4b4b-bc8d-f029490fbea1 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.445151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 32/62] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:17.450891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2d7-f4f7-4976-88d3-6e898289b638 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.451843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2d7-f4f7-4976-88d3-6e898289b638 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.452328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2d7-f4f7-4976-88d3-6e898289b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.481287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9df9d2d7-f4f7-4976-88d3-6e898289b638 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:17.482833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9df9d2d7-f4f7-4976-88d3-6e898289b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:17.522795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f53-8245-4116-af2e-7069c1366d21 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.522795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f53-8245-4116-af2e-7069c1366d21 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.522795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f53-8245-4116-af2e-7069c1366d21 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.522795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 33/63] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:17.529856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f10-c37d-4a6b-b297-3f7519625cf3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.530674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f10-c37d-4a6b-b297-3f7519625cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.531149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906f10-c37d-4a6b-b297-3f7519625cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.544021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:17.565430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bf906f10-c37d-4a6b-b297-3f7519625cf3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:17.566656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bf906f10-c37d-4a6b-b297-3f7519625cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:17.596788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 31/64] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:17.683497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:17.784736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc20e06-e062-4971-973a-70c567860773 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.785384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc20e06-e062-4971-973a-70c567860773 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.786036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc20e06-e062-4971-973a-70c567860773 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.787829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 32/66] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:17.793994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cf026-f937-488c-b106-f6bc51b53c30 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.794285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cf026-f937-488c-b106-f6bc51b53c30 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.794574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cf026-f937-488c-b106-f6bc51b53c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.824984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-415cf026-f937-488c-b106-f6bc51b53c30 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:17.826227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-415cf026-f937-488c-b106-f6bc51b53c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:17.860600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed11c-b269-4ff7-8c26-bde82f353c9b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.860915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed11c-b269-4ff7-8c26-bde82f353c9b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.861151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed11c-b269-4ff7-8c26-bde82f353c9b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.862478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 33/67] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:17.867511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.867825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.868103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.893776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:17.894984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:17.958431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:17.968027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 35/68] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:17.970055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40b32-c84e-4003-a247-b77a457a0932 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.970494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40b32-c84e-4003-a247-b77a457a0932 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.970909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40b32-c84e-4003-a247-b77a457a0932 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:17.973545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 36/69] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:17.979101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:17.979552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:17.979999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.013114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:18.014572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:18.026037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 34/70] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:18.027524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.132984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 37/71] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:18.134547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.152871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf580721-7031-48c1-846c-19d6e708b0b9 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.154840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.194919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.195825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.196612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.197035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.209601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:18.215058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:18.215972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:18.216730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:18.219178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 35/72] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:18.225712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.287001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43da13-e222-432d-bc01-9bc2730c5d4a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.313606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.322684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.323623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.324077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.324558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.336243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:18.341388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:18.342283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:18.343027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:18.346067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 38/73] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:18.347782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.357880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544021b-ff34-4060-a04b-f2e77845ffa7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.398082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.399070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.399362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.399641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.400336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:18.400648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.402033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.414928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 36/74] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.415817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.443868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.460452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43479c-06fc-4e08-9ac1-1b4c8dffb51d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.498400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.499037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.499325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.499605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.510590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:18.515440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:18.516982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:18.517776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 39/75] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:18.518874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.531457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365117a2-5bed-415d-9649-e256e1cd6642 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.569035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.571168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.571819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.572116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.572399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.573075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:18.573387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.574777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.588381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.588381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.621613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663dce-74f8-4186-a607-205c7b4f18c0 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.657499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.658062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.658371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.658651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.659808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.661301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.663109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.664564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.673638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.688004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44721fb5-4f1c-4457-8fd3-de5e80b37a08 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.722630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.723275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.723566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.723878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.725015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.726722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.728267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.728887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.767984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44bda2-caa4-4ca7-b7c5-5dedb230e244 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.782425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.809188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.809835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.810125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.810408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.821394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:18.826533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:18.827390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:18.828183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 41/79] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:18.829297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.846002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51ff1c-1075-45db-81ad-3f399dc23edb None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.880445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.881055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.881349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.881630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.882291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:18.882599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.884016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.892424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 39/80] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.893195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.913835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:18.929479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ea916-a283-46c6-8d54-377974585f48 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:18.964613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:18.965244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:18.966233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:18.966233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:18.979703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:18.981206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:18.982883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 42/81] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:18.983683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:18.999987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238d9dd-5156-463b-ac57-72217cc34ccf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.014013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.034786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.035434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.035750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.036032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.036686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.038133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.083247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71137a4d-9048-4506-a619-104e08129ce4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.118768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.119369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.119677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.119955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.136393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 40/82] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:18 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:19.136393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.219206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.221124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.233384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.258064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.267797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc631ee-ccf8-468b-8789-2c3bbc81021e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.315206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.315861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.317829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.317829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.317829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:19.317829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.318798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.328180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 41/83] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:19.329709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.349690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 43/84] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:19.350676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.464277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1adcf4-5f01-441d-8548-174d64a26992 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.487879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eea32-0d05-4736-9a7a-f9d2c0b62dc5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.508922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.509565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.509851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.510138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.511290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.512800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.514417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 42/85] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:19.515200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.521993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.527594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.537258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.537258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.552530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.552530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.637158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.639134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.644027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fedd01-0171-4160-9ef7-6683107ac718 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.682672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.683481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.683901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.684296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.685603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.687228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.689664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 43/86] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:19.690626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.754373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 44/87] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:19.755660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.771032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.793706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:19.805340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa37f19-8840-4176-99aa-6280c61c39ca None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.849992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.851522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.852058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.852529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.854258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:19.856698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:19.859501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 44/88] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:19.860784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:19.879187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a64-fe10-4a23-98ee-d27c65ed2742 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:19.933498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:19.934225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:19.934651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:19.935118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:19.994127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9ec65-e6a5-4720-b9c8-aeeda394f9ba None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.007899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.009528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.040748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.041609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.042159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.042577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.042912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.059952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.128064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 45/89] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:20.128064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.148331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.150023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.243999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b50c5-24b4-438b-952c-bbc9a4b1cd7f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.279031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.284855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.285607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.286076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.286566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.287858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.289382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.291438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 46/90] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.292351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.292939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 45/91] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:20.294584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.346411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.408378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19827e76-e23a-4209-8e3e-2d6fdd9b3a5d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.416864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad2fef-69b5-408a-b0a1-a3df5b8094ca None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.451041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.451890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.452447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.452692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.453644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=ddb604f3212045e99bbc5f99912931b2) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:20.454013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.455601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.461980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.462603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/role_assignments?user.id=ddb604f3212045e99bbc5f99912931b2&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.462963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.463453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.463799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.464066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.465425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.467444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.476389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 46/93] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.477761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.588193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.614378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243d62-eb50-40cb-9c36-7ce5fa855b4e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.623138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.656233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.658158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.658741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.659160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] PATH_INFO: `/v3/projects/311070b81e99414499b6ac13f668c434` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.660546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] RBAC: Authorizing `identity:get_project(project_id=311070b81e99414499b6ac13f668c434)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.662150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.663992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/projects/311070b81e99414499b6ac13f668c434 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.665046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.668153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84beca-7b0a-48a1-9ca5-9b0ed8cef9f9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.729636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.730263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.730554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.730836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.731579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:20.731925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.733734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.745820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 48/95] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.746673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:20.776427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ede86-84c5-4c34-af4f-15f326ce45e6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.816848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.817857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.818356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.818920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.820009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:20.820570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.822535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7c1c2-78b1-4f43-8602-6a27c184f70d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:20.829964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=7647dfe2bb394802a35379f976d57e94 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:20.952319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f547-b76d-4751-8be7-7e46ba2a24ff None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:20.983806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:20.992539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:20.993168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:20.993457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:20.993739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:20.994485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=311070b81e99414499b6ac13f668c434, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:20.994912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:20.996434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e8127-1034-46dc-8531-8be871609e35 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:21.003344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:20 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=311070b81e99414499b6ac13f668c434&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:21.960817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543ddd-1459-4c82-a44b-f991ab0e016c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:21.961325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543ddd-1459-4c82-a44b-f991ab0e016c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:21.961508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543ddd-1459-4c82-a44b-f991ab0e016c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:21.962875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:21.968347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9654-dfa4-4f89-8c80-20d2c5bb7489 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:21.968782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9654-dfa4-4f89-8c80-20d2c5bb7489 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:21.968978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9654-dfa4-4f89-8c80-20d2c5bb7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:21.992080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-af6d9654-dfa4-4f89-8c80-20d2c5bb7489 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:21.993285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-af6d9654-dfa4-4f89-8c80-20d2c5bb7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:22.067770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:22.104953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbcec4-043d-4a62-a295-f2a9e33d06ff None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:22.105249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbcec4-043d-4a62-a295-f2a9e33d06ff None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:22.105525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbcec4-043d-4a62-a295-f2a9e33d06ff None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:22.107041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 50/100] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:22.113289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:22.113588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:22.113868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:22.132707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:22.133914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:22.229851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 51/101] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:22.478447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbcec4-043d-4a62-a295-f2a9e33d06ff None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:22.529589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:22.608339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbcec4-043d-4a62-a295-f2a9e33d06ff None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:22.643622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:22.644290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:22.644580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:22.644855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:22.646015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:22.647455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:22.649423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 51/102] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:22.656547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:22.720165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:22.786251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540c231-83f0-4eaa-936a-9fffedd0bde1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:22.806114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:22.824170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:22.824835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:22.825122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:22.825406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:22.826075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:22.827538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:22.932252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae5fd-67fb-42f1-adff-50cc16d57b10 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:22.933227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 52/103] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:07:22 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:22.946423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:22.987236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:22.988037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:22.988222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:22.988513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.004269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:23.009475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:23.010388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:23.011149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 52/104] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:23.017466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:23.173469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718c03-6554-4c21-a052-f20c25e18155 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:23.182104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:23.216908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:23.217620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:23.217964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:23.218295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.219001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:23.219907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:23.221527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:23.232382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 53/105] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:23.238946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:23.373051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbee5f-da8b-4ca2-ad49-3fea581793a8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:23.420244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:23.420883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:23.421167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:23.421446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] PATH_INFO: `/v3/users/41d48a092dac4496ab5aae80f404f8e1` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.439358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] RBAC: Authorizing `identity:get_user(user_id=41d48a092dac4496ab5aae80f404f8e1)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:23.440796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd24617-8f58-48e8-b08d-ad312af8aba5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:23.442970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 53/106] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:23 2022] GET /identity/v3/users/41d48a092dac4496ab5aae80f404f8e1 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:23.448623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:23.584096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a988f-7c55-4637-b64a-dd55c3f74664 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:23.634793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:23.635428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:23.635734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:23.636011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.674982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:23.711948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:23.713345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:23.784650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354561-8d98-4df6-887b-849307b9a734 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:23.785029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354561-8d98-4df6-887b-849307b9a734 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:23.785217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354561-8d98-4df6-887b-849307b9a734 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.786560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 54/107] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:23.815612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:23.815923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:23.816205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:23.841728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:23.843118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:23.861137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 54/108] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:23.867223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:23.872349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:23.967909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 55/109] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:23.998420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:24.044382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bad17-0499-4ce8-8ac7-1e3305044053 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:24.088278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:24.089273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.089819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.090296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.091758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:24.093439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913021d-e6b8-474d-b22b-8caa83fc83b0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:24.096231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 55/110] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:23 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:24.101836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:24.144393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872901-cf61-4f9d-ac9a-4f20dd390fa3 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.145030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872901-cf61-4f9d-ac9a-4f20dd390fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.145506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29872901-cf61-4f9d-ac9a-4f20dd390fa3 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.147054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 56/111] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:24.157705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.158301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.158860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.194429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:24.197567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:24.281301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fb27f-b72e-49fc-b93c-b05a7aea0c42 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:24.338773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:24.339409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.339737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.340023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.340895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=41d48a092dac4496ab5aae80f404f8e1) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:24.341220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:24.342688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d34c1-3d6e-4b72-9c8f-f0a8a2b1c81b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:24.348683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 56/112] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:24 2022] GET /identity/v3/role_assignments?user.id=41d48a092dac4496ab5aae80f404f8e1&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:24.350181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c8b4-2a8b-4f43-b7e1-0bd6767f0700 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.350472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c8b4-2a8b-4f43-b7e1-0bd6767f0700 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.350756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c8b4-2a8b-4f43-b7e1-0bd6767f0700 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.352103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 57/113] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:24.359200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e968d4-6ade-4ffc-9fbe-3a971ec6882d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.359508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e968d4-6ade-4ffc-9fbe-3a971ec6882d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.360371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e968d4-6ade-4ffc-9fbe-3a971ec6882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.371695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 57/114] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:24.372635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:24.381803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-27e968d4-6ade-4ffc-9fbe-3a971ec6882d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:24.383046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-27e968d4-6ade-4ffc-9fbe-3a971ec6882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:24.399558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:24.513983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 58/115] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:24.515199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:24.532245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1f17-7ae6-4bcf-bbcd-9d6efe39c45c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.532515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1f17-7ae6-4bcf-bbcd-9d6efe39c45c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.532746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1f17-7ae6-4bcf-bbcd-9d6efe39c45c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.534273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 59/116] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:24.539379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.540685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.540685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.564803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:24.566341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:24.594782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f9d6-510f-4e74-8ce7-b4403c71ede4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:24.681490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:24.682853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.683380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.683901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:24.686037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 60/117] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:24.716764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:24.722190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:24.723047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:24.723867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:24.731767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 58/118] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:24.734695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:24.825831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:24.924043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:24.959044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338741e-d2c2-4d3b-ad3e-671b055a54ed None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:24.986232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:24.997999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:24.998692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:24.998982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:24.999317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.002477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110fa0a-c5da-44e9-99da-bd1335c34b63 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.011129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:25.016200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:25.017112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:25.017915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 59/119] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:25.019430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.066728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.067352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.067643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.067945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.068622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:25.068937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.070605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.079249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 61/120] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.083857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.147289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf6e8-b3b5-44dc-a347-3888b7978421 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.183824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.184473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.184762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.185044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.195875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:25.201122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:25.202052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:25.202817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 60/121] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:25.203538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.209333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b79f-ae43-4177-a9d9-fb1898649b21 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.257400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.260041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.260844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.261459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.262182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.263034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:25.263545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.265683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.277421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.278478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.283056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.336626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea5d5d-c49f-4389-9eda-626ad1f85be9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.373870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.374614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.374975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.375842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.387233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:25.392922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:25.393972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:25.394910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:25.399699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:25.399699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.416010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b83a-fa1f-4794-9356-1fd5771ba486 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.461456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.462165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.462716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.463042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.464472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.466625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.471812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.472848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.515776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977149e7-0c6f-4a7e-b3f8-891237ffec0e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.519879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.556733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.557933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.558492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.559008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.559857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:25.560517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.562736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.570802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.575922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 62/125] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.577015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.610491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb3ad7-c0ad-4ca5-a42e-e41ec1a4e7fd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.659055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.660072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.660775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.661197 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.663182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.665460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.667466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.668663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.725424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e78c-0a64-49af-99b5-5a294dbc268f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.768716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.769835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.771228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.771717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.772539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:25.772989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.774513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.787309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.792761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.809040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4996b8b-a7e5-4061-986d-b06daa1ef1cf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.834803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.849193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.850052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.850490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.850879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.852239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.853881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.855717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.860080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:25.868908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:25.943875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab90d4-3c1e-46d6-ad21-33456139819e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.968433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b996-1a61-49ee-aa2b-3e2974a993c6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:25.988356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:25.989249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:25.990064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:25.990484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:25.992102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:25.994446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:25.996691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 64/129] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:25.997657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.009811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.010594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.010991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.011669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.012827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.014613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.017120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:26.018372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.098228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.119740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4dff2-4f35-420a-a153-d2d80ce6a79a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.125787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bb43c-fdef-41ac-bd0f-09b4a39b524b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.127994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.156079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.161964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.161964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.161964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.161964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.161964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.162702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.163320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:25 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:26.167740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.167740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.167740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/ddb604f3212045e99bbc5f99912931b2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.169278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.170306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, user_id=ddb604f3212045e99bbc5f99912931b2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.171788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.251396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 67/132] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:26 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/ddb604f3212045e99bbc5f99912931b2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:26.256771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.286847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51b3a2-d268-451f-9ce5-386c383fd1bd None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.325818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.327407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.327925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.329012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] PATH_INFO: `/v3/projects/311070b81e99414499b6ac13f668c434` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.331698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] RBAC: Authorizing `identity:get_project(project_id=311070b81e99414499b6ac13f668c434)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.335411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 66/133] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:26 2022] GET /identity/v3/projects/311070b81e99414499b6ac13f668c434 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:26.337051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.384336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.427984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab51bd-1044-4247-8963-e00ba50038f9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.480203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.480878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.481173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.483711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/e7e41ce832594a9d94ee9f12464ad846/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.485604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, user_id=e7e41ce832594a9d94ee9f12464ad846, role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.487592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.489649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f62f4-413a-4691-aaf2-614df460499c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.490137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.526525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.527190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.527479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.527787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.528967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.530696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cf530-a932-42a2-85c9-a196dab8b2b4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.532413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 67/134] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:26 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:26.533564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d375332-3659-4d2e-9866-83f5578a45cf None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.533856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d375332-3659-4d2e-9866-83f5578a45cf None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.534138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d375332-3659-4d2e-9866-83f5578a45cf None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.535559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 68/135] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:26.536165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d375332-3659-4d2e-9866-83f5578a45cf None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.594677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 68/136] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:26 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/e7e41ce832594a9d94ee9f12464ad846/roles/7647dfe2bb394802a35379f976d57e94 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:26.599700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:26.696015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.708956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d375332-3659-4d2e-9866-83f5578a45cf None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.722811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdc434-f1d6-48ba-b8b1-a45192b1d0a3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:26.765720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.767287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.767877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.768387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/e7e41ce832594a9d94ee9f12464ad846/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.770675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, user_id=e7e41ce832594a9d94ee9f12464ad846, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.772739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf50c3-5731-4480-805c-8ad4511e0e33 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.789715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:26.790849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.791322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.792102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] PATH_INFO: `/v3/projects/311070b81e99414499b6ac13f668c434/users/e7e41ce832594a9d94ee9f12464ad846/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.794722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=311070b81e99414499b6ac13f668c434, user_id=e7e41ce832594a9d94ee9f12464ad846, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:26.796607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b069f-678b-404d-9a99-ae10a7fd63a7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:26.847213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 69/137] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:26 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/e7e41ce832594a9d94ee9f12464ad846/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:26.848940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d72-8472-4d77-84d9-d00b61fa710b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:26.849594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d72-8472-4d77-84d9-d00b61fa710b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:26.850372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d72-8472-4d77-84d9-d00b61fa710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:26.877279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0f678d72-8472-4d77-84d9-d00b61fa710b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:26.878789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0f678d72-8472-4d77-84d9-d00b61fa710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:26.904204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 69/138] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:26 2022] PUT /identity/v3/projects/311070b81e99414499b6ac13f668c434/users/e7e41ce832594a9d94ee9f12464ad846/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:26.959026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:26.990654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 70/139] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:27.082148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa6a65-df9d-41cb-a1e9-32f142ee72e3 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.082786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa6a65-df9d-41cb-a1e9-32f142ee72e3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.083308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa6a65-df9d-41cb-a1e9-32f142ee72e3 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.085219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 70/140] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:27.091280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.091563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.091867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.117884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:27.119205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:27.219880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 71/141] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:27.405553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa6a65-df9d-41cb-a1e9-32f142ee72e3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:27.499991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:27.540928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa6a65-df9d-41cb-a1e9-32f142ee72e3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:27.600997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:27.601943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.602412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.602868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.608484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:27.610548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:27.612394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 71/142] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:27.617547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:27.635516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d66a0-4911-4d62-95af-f06ce73767b5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:27.663318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:27.677004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:27.688206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:27.689340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.690706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.691267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.710733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:27.716831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:27.717716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:27.718471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:27.726399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 72/143] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:27.733479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:27.841489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaeff1-b8ee-4bb2-b772-5502249f6a7c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:27.901807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992d3ce-a6db-4b00-9b9d-cf0672c160dd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:27.933017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:27.933658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.933948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.934230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.934916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:27.936498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:27.939484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:27.940126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:27.940427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:27.940708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:27.941390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:27.941700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:27.943076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:27.952100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 73/144] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:27.959956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:27.997011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:28.064994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5329-d5b3-49a0-ab9d-da17736bfe61 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:28.075015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 72/145] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:27 2022] POST /identity/v3/users => generated 283 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:07:28.113002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:28.113002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:28.113002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:28.113002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] PATH_INFO: `/v3/users/41d48a092dac4496ab5aae80f404f8e1` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:28.113346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] RBAC: Authorizing `identity:get_user(user_id=41d48a092dac4496ab5aae80f404f8e1)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:28.113346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:28.113346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 74/146] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:27 2022] GET /identity/v3/users/41d48a092dac4496ab5aae80f404f8e1 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:28.115105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:28.220806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:28.239016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571bef-de9a-44e8-a6d3-2bfbb2d7e0fb None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:28.286870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:28.288351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:28.288727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:28.289006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:28.290272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:28.291762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71e915-61d0-4284-80c8-5102e5116953 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:28.293434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 73/147] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:28 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:28.300068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:28.411465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203ace4-5893-46b2-aa06-414dee6ae34d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:28.414016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:28.449153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:28.449897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:28.450189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:28.450467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/41d48a092dac4496ab5aae80f404f8e1/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:28.452557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, user_id=41d48a092dac4496ab5aae80f404f8e1, role_id=c868f2c45e5d4238bfd9c1c8cad4d5e5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:28.454053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33d1b1-8a5f-40eb-8bf6-56d4ca1360aa None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:28.532391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 75/148] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:28 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/users/41d48a092dac4496ab5aae80f404f8e1/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:29.123055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385c30-72ee-47aa-9856-13032413aea6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:29.123549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385c30-72ee-47aa-9856-13032413aea6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:29.123680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385c30-72ee-47aa-9856-13032413aea6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:29.125767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 74/149] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:29.130993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:29.131282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:29.131559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:29.154276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:29.156040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:29.226592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 76/150] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:29.526715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb385c30-72ee-47aa-9856-13032413aea6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:29.642860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb385c30-72ee-47aa-9856-13032413aea6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:29.693069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:29.693777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:29.694063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:29.694336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:29.709266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:29.714473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:29.715284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:29.716080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 75/151] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:29.724091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:29.870119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:29.877134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb88c9-a19c-403e-b1b3-ff6ad70c15e4 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:29.915333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:29.915975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:29.916257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:29.916598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:29.917299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:29.917595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:29.919180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:29.930047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 77/152] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:29.935751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:29.972031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:30.099307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad1be9-6def-4d90-b637-f9aa7b57bbc3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:30.152738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:30.153343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.153626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.153911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] PATH_INFO: `/v3/users/ddb604f3212045e99bbc5f99912931b2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.155587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] RBAC: Authorizing `identity:get_user(user_id=ddb604f3212045e99bbc5f99912931b2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:30.157230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035f8e3-d41c-4d78-b698-67011eaee546 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:30.158977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 76/153] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:29 2022] GET /identity/v3/users/ddb604f3212045e99bbc5f99912931b2 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:30.165153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:30.304940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40df90-9733-4b4e-9772-983465e9d31a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:30.309250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947b32-d044-4364-91ba-04490e63ba4d None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.309556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947b32-d044-4364-91ba-04490e63ba4d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.309809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947b32-d044-4364-91ba-04490e63ba4d None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.311355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 77/154] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:30.317101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7079b-48f5-4d8d-a1c7-5045c5716729 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.317542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7079b-48f5-4d8d-a1c7-5045c5716729 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.317665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7079b-48f5-4d8d-a1c7-5045c5716729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.351047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:30.351679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.351973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.352252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.361119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bda7079b-48f5-4d8d-a1c7-5045c5716729 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:30.362355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bda7079b-48f5-4d8d-a1c7-5045c5716729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:30.385709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:30.444804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:30.446245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbc67b-155f-4b3e-aa18-c6151c597971 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:30.515291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 78/155] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:30.516827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e2a9-fffe-49e7-ae6b-2bf46899a422 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.517125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e2a9-fffe-49e7-ae6b-2bf46899a422 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.517407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e2a9-fffe-49e7-ae6b-2bf46899a422 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.518792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 79/156] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:30.533349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89077-2a6e-4054-a689-966dc1325315 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.533645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89077-2a6e-4054-a689-966dc1325315 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.533927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89077-2a6e-4054-a689-966dc1325315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.585479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bae89077-2a6e-4054-a689-966dc1325315 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:30.586693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bae89077-2a6e-4054-a689-966dc1325315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:30.593067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:30.602568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 78/157] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:30.603977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6886ce-305e-407b-84c6-5bf762e29b1f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.608895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6886ce-305e-407b-84c6-5bf762e29b1f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.608895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6886ce-305e-407b-84c6-5bf762e29b1f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.608895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 79/158] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:30.612583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6886ce-305e-407b-84c6-5bf762e29b1f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:30.727913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:30.732932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 80/159] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:30.734985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.735275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.735552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.770161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:30.771372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:30.819701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6886ce-305e-407b-84c6-5bf762e29b1f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:30.867259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:30.868765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:30.868765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:30.868765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:30.870957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:30.874417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:30.878446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 80/160] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:30 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:30.884432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:30.917907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 81/161] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:31.028147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b11d2-c580-4988-b2ca-afea5aebdb31 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:31.059095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:31.070038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:31.070746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:31.071036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:31.071321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:31.072083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=ddb604f3212045e99bbc5f99912931b2) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:31.080409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:31.080409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:31.084229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 81/162] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:30 2022] GET /identity/v3/role_assignments?user.id=ddb604f3212045e99bbc5f99912931b2&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:31.209992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:31.312387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:31.349782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f387-4f1c-4745-8dc4-778659fb638a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:31.391973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:31.392968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:31.393451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:31.393921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:31.415788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:31.421027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:31.422027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:31.423362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:31.425958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:31.431884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:31.532851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:31.643905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3001832-4ab4-42cf-9204-5dc6235f9229 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:31.695150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:31.695801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:31.696089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:31.696367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:31.697044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:31.697355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:31.698737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:31.707504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 82/164] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:31.715096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:31.733826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3f2e4-48a1-48d0-89f1-2f158baf340b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:31.781703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:31.798725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:31.824457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:31.825515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:31.825973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:31.826411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:31.848958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:31.854445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327723-2d4c-492c-9e2f-adbb56aa58a7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:31.860133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:31.860981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:31.861746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:31.864721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 83/165] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:31.870446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:31.902362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:31.902917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:31.903201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:31.903477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:31.904638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:31.906319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:31.912682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 83/166] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:31.913536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.068987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdfbb-5a60-44c8-a8c6-bf23362fc891 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.108379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc25579-1b2e-45a2-bda5-9808c35d08f0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.124890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.126287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.126741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.127215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.128080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:32.128570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:32.130312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:32.144805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:32.150424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 84/167] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:32.151751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.175284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.176826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.177368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.177818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.197090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:32.209588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:32.210588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:32.211383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:32.213827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 84/168] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:32.214848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.240038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:32.326266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ece8d9-852a-4e98-a408-1bd8b7475d39 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.388154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.390261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728cade-9335-4bee-8f1b-76a829ed0062 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.391819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.393748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.394211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.460670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.461222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.461501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.461778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.464531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:32.465995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:32.475837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 85/169] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:32 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:32.476767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.526231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:32.527742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e75417-6b7a-4ccc-ba92-3cc1008de495 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:32.568014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:32.626580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:32.674946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1eaf5-8152-4879-805a-d3b6dc955ea5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.712496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 85/170] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:32.713621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c84ff-de95-41f7-bc70-c34462d1ccd6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.714701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c84ff-de95-41f7-bc70-c34462d1ccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.714992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c84ff-de95-41f7-bc70-c34462d1ccd6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.717098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 86/171] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:32.717924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c84ff-de95-41f7-bc70-c34462d1ccd6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.738579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.739210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.739496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.739838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.740517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:32.740885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:32.743615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:32.756414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 86/172] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:32.757630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:32.918871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559607-5c58-4157-b7ba-9bf94953b9fe None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.937091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c84ff-de95-41f7-bc70-c34462d1ccd6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:32.944025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:32.970690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:32.971764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.972283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.972822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:32.974388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:32.976368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad46c-9be1-4ac2-bc81-0b81aa5970ac None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:32.978271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 87/173] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:32 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:32.979675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:32.980353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:32.980891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.014682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.015311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.015594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.016414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:33.016531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.017935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:33.046546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.097403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.098007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 88/174] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:33.099016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.099394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.207359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50760f5-c31f-4547-8a52-6225e3c62c82 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.208632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 87/175] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:33.209643 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.216530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.247640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.247903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.248281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.248665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] PATH_INFO: `/v3/users/e7e41ce832594a9d94ee9f12464ad846` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.250043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] RBAC: Authorizing `identity:get_user(user_id=e7e41ce832594a9d94ee9f12464ad846)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.251726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb268c-0bcf-49c2-9ce3-a30066cf4afc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.253602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 89/176] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/users/e7e41ce832594a9d94ee9f12464ad846 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:33.255111 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4109c-88d4-436d-b66c-c173299a5234 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.255971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4109c-88d4-436d-b66c-c173299a5234 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.256685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4109c-88d4-436d-b66c-c173299a5234 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.258713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 90/177] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:33.259539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4109c-88d4-436d-b66c-c173299a5234 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.270975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.323951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7ca74-25d1-407e-8ffc-c05110a3afc8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.361833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.362763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.363327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.363962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.364924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:33.365404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.367368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.371884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4109c-88d4-436d-b66c-c173299a5234 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.374897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 88/178] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=7647dfe2bb394802a35379f976d57e94 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:33.376778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.442673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.443375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.443678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.443995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] PATH_INFO: `/v3/projects/311070b81e99414499b6ac13f668c434` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.451700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=311070b81e99414499b6ac13f668c434)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.451700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea250-8ac7-4d5e-9a90-29d55b189bf1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.452163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 91/179] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/projects/311070b81e99414499b6ac13f668c434 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:33.453975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.454269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.454549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.499915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:33.499915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:33.584412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad027c0-2ee6-46d7-85c1-e0cf4c758e81 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.588054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.608066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.618782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 92/180] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:33.619696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.642700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.643480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.643885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.644302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.727902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.729654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd17dde-22da-45d8-807d-c9d2b0d61c8a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.748024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8f668-8c96-4782-be16-95997f2e46f1 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.788776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.789438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.789662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.789937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.790881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=311070b81e99414499b6ac13f668c434, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:33.791506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:33.793007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f3099-1eda-4458-b5d9-ae0392635682 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:33.799556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 93/181] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=311070b81e99414499b6ac13f668c434&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:33.800787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba9321-1c86-4be5-9158-ef42aad586b8 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.801080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba9321-1c86-4be5-9158-ef42aad586b8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.801361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba9321-1c86-4be5-9158-ef42aad586b8 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.802807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 94/182] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:33.803445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba9321-1c86-4be5-9158-ef42aad586b8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.849802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.852346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 89/183] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:33.857886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987a03f-6ea9-4969-bbf7-6514546ae9ff None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.857886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987a03f-6ea9-4969-bbf7-6514546ae9ff None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.857886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987a03f-6ea9-4969-bbf7-6514546ae9ff None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.863899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 90/184] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:33.863899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.863899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.863899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:33.893208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:33.894448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:33.907252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:33.946503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba9321-1c86-4be5-9158-ef42aad586b8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:33.976451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 91/185] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:33.983947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:33.996001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:33.997045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:33.997334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:33.997751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.010563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-eb7e2970-82f6-475e-ac10-866cd64e6746 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:34.016064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:34.016965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:34.017737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 95/186] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:34.018839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.018947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.019229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.037622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:34.039610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:34.102721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad92d-93dc-4cca-8e62-82cfc53b12d9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.122499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 96/187] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:34.123364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.157406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.159101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.159623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.160358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.160817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.163167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.164976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.167190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 92/188] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:33 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.168237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.189184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.265482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19f400-7008-4e14-b0d5-4b918654d840 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.291665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24b3bc-457c-4d73-9e54-9fb94813c3c9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.304110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.305018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.305503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.305967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.317321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:34.322360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:34.323430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:34.324311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:34.324909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 97/189] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:34.330705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.331767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.332384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.332672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.332950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.333616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:34.333921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.335310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.348253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 93/190] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.348253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.422846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.429909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.437472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4ab5-21da-409f-a1cc-a8f35e518111 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.451802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e21a2-ded5-4601-a120-b1eb36058409 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.472199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.472814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.473101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.473402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.474195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=e7e41ce832594a9d94ee9f12464ad846) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:34.474513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.475978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.481734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 98/191] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/role_assignments?user.id=e7e41ce832594a9d94ee9f12464ad846&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.482528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.485615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.486265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.486552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.486835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.488077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.489653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.492578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 94/192] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.493542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.607719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78832d3f-903d-4283-a6be-4a61cc569bf6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.616669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecf7c2-6098-44ba-adc7-c35793f78ff6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.648258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.648890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.649281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.649450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.650126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:34.650437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.652513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.655868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.657355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.657652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.657932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] PATH_INFO: `/v3/users/41d48a092dac4496ab5aae80f404f8e1` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.659676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=41d48a092dac4496ab5aae80f404f8e1)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.661137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.662745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 95/193] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/users/41d48a092dac4496ab5aae80f404f8e1 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.663508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.666881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 99/194] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:34.667888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.685256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.685888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.776731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a064ca6-2f37-44a4-a083-69fcf93fb53d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.783137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f2ee-e1cf-49b9-8656-7fbb0b681bc3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.810408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.811013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.811300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.811578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.812252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.813700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:34.816197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.816829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.817116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.817402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.827681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:34.833514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:34.834555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 96/195] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:34.835460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.904749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:34.934598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd91-9198-48ac-8bcd-94c407cbe5ce None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:34.936468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 100/196] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:34 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:34.936468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:34.974659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:34.975459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:34.975778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:34.976168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:34.991532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:34.993484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.000779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 97/197] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:35.000779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.017277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.055535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8618d-a86d-421b-8e7e-2245bb25544c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.101714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.102747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.103175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.103682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.112085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bf435-7dec-4bf4-9728-37b3a687727f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.150214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.154356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.155030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.156145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.156145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.156503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:35.156830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.158206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.169521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 98/198] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:35.170295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.196932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.197805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.294805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.297725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f570-5df0-4eca-bd4d-36ffa13f6644 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.340105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.340541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 101/199] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:35.340802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.341094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.341373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.341473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.415596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.426616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.428190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.461312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968825d-a301-417e-9c01-44ae1ffba108 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.499284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.499965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.500890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.501173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.502349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.504842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.506785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 102/200] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:35.507638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.537995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 99/201] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:35.538986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.572100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.625900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a169b5-752d-4b78-aa07-bfafebc8ea7d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.644499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.650986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b6cf-957c-4720-825e-b092e35817bc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.666332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.666909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.667262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.667732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.668943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.670429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.672863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 103/202] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:35.674114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.696560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.697191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.697621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.697795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.792959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7768-208e-4ef0-bc1d-483d7d18bf69 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:35.796526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.823359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.825207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.834570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:35.835252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:35.835550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:35.836221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:35.837073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:35.838721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:35.840438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 104/203] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:35.841423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.941288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:35.955784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 100/204] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:35.959796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:35.979406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d971-ff59-4fa8-9cae-85d60084384a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:36.021054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:36.022033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:36.022510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:36.022995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:36.023968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3, user.id=41d48a092dac4496ab5aae80f404f8e1) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:36.024516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:36.026308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:36.032933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 105/205] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/role_assignments?user.id=41d48a092dac4496ab5aae80f404f8e1&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:36.034103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:36.075141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:36.093493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854d98-946e-4496-8452-be2d45202833 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:36.150829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:36.152060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:36.153561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:36.154567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:36.156342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=046cc2b64084423eb14f50f649e7677f) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:36.158524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:36.161177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:36.171405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d66ff-0993-48bf-8f3a-337f1447b3db None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:36.174100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 101/206] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:35 2022] GET /identity/v3/role_assignments?user.id=046cc2b64084423eb14f50f649e7677f&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:36.235283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:36.236419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:36.236915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:36.237472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:36.239501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:36.241602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c906c8-263b-47ee-b2d8-9afaaabbfca4 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:36.246718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 106/207] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:36 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:36.253720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:36.330262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:36.384625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23646a-865f-45fb-b0f9-92c8b327ade5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:36.432356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:36.432970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:36.433251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:36.433525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:36.434252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=046cc2b64084423eb14f50f649e7677f) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:36.434557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:36.436034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04b66e-7f1b-4cb0-b541-f68cf0419eb0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:36.442556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 102/208] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:36 2022] GET /identity/v3/role_assignments?user.id=046cc2b64084423eb14f50f649e7677f&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:37.401429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f07cd-c793-4053-b3c5-cbaa144cc9aa None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:37.401948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f07cd-c793-4053-b3c5-cbaa144cc9aa None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:37.402052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f07cd-c793-4053-b3c5-cbaa144cc9aa None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:37.403447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 107/209] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:37.408874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:37.409367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:37.409440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:37.427852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:37.429024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:37.470081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b9df-8293-4c9c-b04b-04a4d0a15752 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:37.470363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b9df-8293-4c9c-b04b-04a4d0a15752 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:37.470638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b9df-8293-4c9c-b04b-04a4d0a15752 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:37.472036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 108/210] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:37.477714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:37.478008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:37.478319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:37.503932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:37.505270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:37.507143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 103/211] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:37.576224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 109/212] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:37.782785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:37.863361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:37.866227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:37.898400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:37.921237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2553c-0959-445a-b2ec-27b96795045a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:37.958126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:37.959160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:37.959634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:37.960168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:37.969355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c728c3f-1a9e-45f8-8d61-5eb3b0c2fe9b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:37.971764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:37.977104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:37.978061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:37.978849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:37.987112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 104/213] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:37.988320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.006289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.006936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.007246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.007526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.008746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.010225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.011818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 110/214] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:38.017640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.093796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6d3b8-822f-49c9-af83-b1aeb6b7cfba None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:38.112012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:38.123278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e6df4-23ef-4103-b425-55da68ed177d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:38.128079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:38.130386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.131305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.131833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.132316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.133169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:38.133666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.135299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.145136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 105/215] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:38.150931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.162041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.162846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.163035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.163290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.164047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.165593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.263206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79890d2d-c8fd-4f17-9f31-ef84d4293213 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:38.288451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 111/216] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:38 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:07:38.319806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.320918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.321399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.321821 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] PATH_INFO: `/v3/users/9ec4ac3d03d64456a7da18f064ae4da9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.339181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=9ec4ac3d03d64456a7da18f064ae4da9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.341201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5449c-0872-4f38-ac3b-295d262bbcd4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.343735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 106/217] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:38 2022] GET /identity/v3/users/9ec4ac3d03d64456a7da18f064ae4da9 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:38.352745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.485825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bb40-011d-4ffd-9216-835c523b70bb None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:38.514221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0696-ec84-46a4-bf63-e38bde33290a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.514766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0696-ec84-46a4-bf63-e38bde33290a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.515078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0696-ec84-46a4-bf63-e38bde33290a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.517035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 107/218] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:38.523198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57918062-661d-4c2b-a552-f49a0079f093 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.523484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57918062-661d-4c2b-a552-f49a0079f093 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.523820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57918062-661d-4c2b-a552-f49a0079f093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.531140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.532069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.532540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.532817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.548392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-57918062-661d-4c2b-a552-f49a0079f093 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:38.550592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-57918062-661d-4c2b-a552-f49a0079f093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:38.567794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:38.600721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:38.634355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.634355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.660206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 108/219] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:38.698668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20decdef-45d4-4ab8-aea2-e8cad2ffdc08 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.699048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20decdef-45d4-4ab8-aea2-e8cad2ffdc08 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.699333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20decdef-45d4-4ab8-aea2-e8cad2ffdc08 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.701063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 109/220] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:38.708431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.708431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.708431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.730528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:38.732956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:38.761668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 112/221] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:38.769202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.827823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 110/222] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:38.848148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:38.893404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb23bf-1eda-48ab-a8b7-ccab4f6a7782 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:38.938043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:38.938698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:38.938980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:38.939255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:38.944355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:38.945873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:38.947609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 113/223] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:38 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:38.954275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:38.991565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.096489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34871-c8c2-419f-864c-6c6e484cf21c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.124496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:39.131752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:39.133992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.134894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.135376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.135839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.145669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f948-4fb8-41ab-80fe-e6cce58a4f58 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.153817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:39.164120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:39.165042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:39.165825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:39.172303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 114/224] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:39.180694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.215669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.216591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.217055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.217513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.218452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=9ec4ac3d03d64456a7da18f064ae4da9) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:39.219919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:39.223229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:39.236071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 111/225] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:38 2022] GET /identity/v3/role_assignments?user.id=9ec4ac3d03d64456a7da18f064ae4da9&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:39.265012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.411328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e604acf-dc57-4759-a88a-bd8e8c237484 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.426785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04435e-296d-4c72-94ee-a42899a502c5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.476734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.477771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.478261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.478741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.480471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:39.480645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:39.485100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:39.489054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.495699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.496315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.496841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.502103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 115/226] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:39.507962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.522460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:39.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:39.528414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:39.529202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 112/227] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:39.535077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.552684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:39.635743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:39.651259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808533a-b3e3-439a-8edf-406c145cbe4c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.690557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.691092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.691377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.691669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.692515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:39.692873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:39.694735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:39.705140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 113/228] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:39.711211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.796027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:39.813645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c1d70-06fa-497f-8cd7-d4dd8f9f9cf1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.819916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3826c9e-e40f-46e1-8ea9-b5d0882e2af3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:39.858784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.859537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.860102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.860382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.861532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:39.863192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:39.865468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 114/229] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:39.871110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:39.906179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:39.907361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:39.907640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:39.916399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:39.918734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:39.920249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:39.928695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 116/230] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:39.929624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:40.097668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852859-75a7-4812-b2e6-e9dbd78e2a2c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:40.098891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1d02d-1a38-4f2a-a649-88479a79dbdf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:40.117931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:40.146382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:40.147031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:40.147313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:40.147588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:40.149418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:40.150935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:40.153374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 117/231] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:40.161193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:40.177860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:40.178519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:40.178903 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:40.179192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:40.182076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:40.183702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:40.185484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 115/232] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:39 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:40.191279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:40.220559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:40.272693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51452b-cdbd-4625-ada3-01ddd3ba3155 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:40.324382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:40.326808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:40.327300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:40.327836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/046cc2b64084423eb14f50f649e7677f/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:40.331989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, user_id=046cc2b64084423eb14f50f649e7677f, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:40.333933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b680f2-3bcf-43c6-8467-6ebae49fe53c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:40.382592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8ca00-953c-4f46-92f8-f0fd3bb658b9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:40.411366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 118/233] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:40 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/046cc2b64084423eb14f50f649e7677f/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:40.428101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:40.429185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:40.429681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:40.430167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/046cc2b64084423eb14f50f649e7677f/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:40.433181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, user_id=046cc2b64084423eb14f50f649e7677f, role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:40.435125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18643a-95ca-41ef-8800-276869091f27 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:40.525920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 116/234] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:40 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/046cc2b64084423eb14f50f649e7677f/roles/7647dfe2bb394802a35379f976d57e94 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:07:41.128968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ccb7-44d1-4a51-807b-64e5d41fbbdb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.129491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ccb7-44d1-4a51-807b-64e5d41fbbdb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.129582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ccb7-44d1-4a51-807b-64e5d41fbbdb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.131370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 119/235] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:41.136565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.136856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.137138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.156067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:41.157794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:41.213557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45dc67-daff-4d4c-9ff3-412f168da7e8 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.213851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45dc67-daff-4d4c-9ff3-412f168da7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.214045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45dc67-daff-4d4c-9ff3-412f168da7e8 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.215387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 120/236] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:41.221146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.221448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.221714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.240463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:41.241658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:41.276041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 117/237] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:41.292116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:41.320979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 121/238] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:41.633870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:41.636529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:41.706664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:41.808159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39c0ba-737b-47dc-9873-0b0f539893e2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:41.844961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:41.845602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.845885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.846158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.857278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:41.861072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc952-84e4-4df0-8a1b-c29bfaab307e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d27ec91e-f1fc-44a8-ada7-276ed23b8507 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:41.862750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:41.863929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:41.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 118/239] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:41.865606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9681-aec2-447d-9e57-ddc2056faac7 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.865606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9681-aec2-447d-9e57-ddc2056faac7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.865606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9681-aec2-447d-9e57-ddc2056faac7 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.870863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 119/240] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:41.873070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c9681-aec2-447d-9e57-ddc2056faac7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:41.916406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:41.916980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.917255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.917532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.918651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:41.920304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43726fd9-07b1-47e5-8ac8-0476a93fa244 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:41.928374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 122/241] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:41.930106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:41.930403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:41.930690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:41.951942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:41.953950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:41.954473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:41.996337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:42.002397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c9681-aec2-447d-9e57-ddc2056faac7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.039749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.040401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.040598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 123/242] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:42.040698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.040947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.041629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:42.041937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:42.042548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.043345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:42.055702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 120/243] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:42.062291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.238489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57cc9d-ba89-4a60-ab8a-b152d83703ba None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.251538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589bf4a-bf35-479f-a346-7bcf1310acb6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.291215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:42.305834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.306461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.306750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.307029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.307713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:42.309153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:42.313581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.314220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.314506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.314784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] PATH_INFO: `/v3/users/101bdd5543a24f15a91363d45c8f6655` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.336487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] RBAC: Authorizing `identity:get_user(user_id=101bdd5543a24f15a91363d45c8f6655)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:42.337963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:42.339631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 121/244] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/users/101bdd5543a24f15a91363d45c8f6655 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:42.345730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.416399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 124/245] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:42 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:42.418031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.478502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:42.493040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d85b99-5bf2-4d6e-a7e0-009efb7a080b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.570839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.571476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.571797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.572074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.628256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e958be-3bae-444c-80b9-ccf69c6ff5d0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.672918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:42.684408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.685655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.687683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.687683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.698616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:42.702141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:42.709183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:42.714500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:42.715416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:42.716256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:42.716868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 125/246] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:42.726553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.871145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:42.883268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 122/247] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 19:07:42.885543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877f2c1-beb4-4bc0-8756-9963c51c3e03 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:42.890165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:42.933582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:42.935149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:42.935719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:42.936212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:42.937118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:42.937675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:42.939302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:42.950839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 126/248] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:42.957990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:43.003143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:43.037687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1887e-d709-49d2-bfe8-1b0f62a4aab2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:43.093981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:43.094627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.095069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.095498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.100257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:43.102013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:43.104176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 123/249] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:43.111345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:43.117844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41b89-1d61-4687-943d-531950622f06 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:43.198662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:43.199237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.199526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.199838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] PATH_INFO: `/v3/users/9ec4ac3d03d64456a7da18f064ae4da9` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.201041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] RBAC: Authorizing `identity:get_user(user_id=9ec4ac3d03d64456a7da18f064ae4da9)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:43.202761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebc79-f601-48f8-9879-c820a91a0e1a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:43.204650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 127/250] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:42 2022] GET /identity/v3/users/9ec4ac3d03d64456a7da18f064ae4da9 => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:43.206973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd443bb-f885-4b15-aefc-5e9a20a76c7f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.207266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd443bb-f885-4b15-aefc-5e9a20a76c7f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.207549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd443bb-f885-4b15-aefc-5e9a20a76c7f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.209212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 128/251] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:43.214220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd443bb-f885-4b15-aefc-5e9a20a76c7f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:43.241876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f197c-2754-48e2-a14f-f946736b3109 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:43.246673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:43.284438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:43.285423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.285900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.286390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.287948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=101bdd5543a24f15a91363d45c8f6655) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:43.288774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:43.291121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0e08-cc68-4ea3-b2d2-177be6400b42 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:43.299826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 124/252] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:43 2022] GET /identity/v3/role_assignments?user.id=101bdd5543a24f15a91363d45c8f6655&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:43.301408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576519-4e7c-49a7-9911-c1396545fa0d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.301913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576519-4e7c-49a7-9911-c1396545fa0d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.302379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576519-4e7c-49a7-9911-c1396545fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.324599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0f576519-4e7c-49a7-9911-c1396545fa0d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:43.326133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0f576519-4e7c-49a7-9911-c1396545fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:43.339135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:43.379891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd443bb-f885-4b15-aefc-5e9a20a76c7f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:43.458754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 125/253] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:43.461001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103454-b881-426a-8617-c952f837ad57 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.461293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103454-b881-426a-8617-c952f837ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.461571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103454-b881-426a-8617-c952f837ad57 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.464388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 126/254] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:43.465071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:43.465734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.466019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.466298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.472563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:43.474423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:43.476288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 129/255] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:43 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:43.478995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.479285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.479560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.483565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:43.515247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:43.517006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:43.622231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 127/256] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:43.668014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:43.688228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760b3d1-3b2b-490d-85ad-a80d70e7bcaf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:43.737896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:43.738612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:43.738900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:43.739182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/9ec4ac3d03d64456a7da18f064ae4da9/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:43.748985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, user_id=9ec4ac3d03d64456a7da18f064ae4da9, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:43.750500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:43.834511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 130/257] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:43 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/9ec4ac3d03d64456a7da18f064ae4da9/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:43.841983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:43.962539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:43.996326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dbfd1-3bab-402d-aca4-3d55c2db7bb6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.046199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.046900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.047182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.047457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:44.061228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:44.066633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:44.067515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:44.068337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 128/258] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:07:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:44.073828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.203527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.261928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6573350-4714-48ac-9197-47b0c4dbdace None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.355720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.356347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.356627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.356898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:44.357565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:44.357867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:44.359769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:44.376500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 131/259] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:44.388707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.402661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:44.450437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628881-d0d7-4b80-908d-d701d691b5ab None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.537945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.538889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.539180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.539462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:44.548164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:44.565576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9e31-ad9a-4ef4-a276-9a123b14509d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.570769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:44.579675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:44.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:44.581486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 129/260] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:44.584671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.620948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.621946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.622404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.622892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:44.625351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:44.626998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:44.629025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 132/261] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:44.637000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.804797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f07a-9f3e-4333-bbf2-0df2dc866d27 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.830275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:44.887474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17770400-30b9-43a6-a081-6f7ed82c7db3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:44.889349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.889992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.890282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.891339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:44.891941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:44.892253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:44.893650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:44.917733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 130/262] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:44.924059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:44.964438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:44.965123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:44.965419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:44.965698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.080878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:45.081053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.083633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.177389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:45.181233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1749fd-0dd8-4f29-83fd-18347a7994d6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:45.202499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 133/263] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 19:07:45.212652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:45.234669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:45.235438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:45.239670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:45.239670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] PATH_INFO: `/v3/users/046cc2b64084423eb14f50f649e7677f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.239670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] RBAC: Authorizing `identity:get_user(user_id=046cc2b64084423eb14f50f649e7677f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.239670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.248803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 131/264] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:44 2022] GET /identity/v3/users/046cc2b64084423eb14f50f649e7677f => generated 283 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:45.254918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:45.391106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc02d30-5812-4d93-8bc8-b72cf6b8e227 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:45.418473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:45.429123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:45.431886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:45.431886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:45.431886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.441893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5f94-bd17-4bcd-9185-733a2edf8d93 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:45.502060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.503565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.525385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:45.526042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:45.526337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:45.526620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.532377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.533837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.535919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 134/265] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:45 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:45.541978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:45.609669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 132/266] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 19:07:45.620027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:45.625756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:45.668899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:45.739230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2676-96d8-4aef-a3c9-5482ed53f748 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:45.808378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:45.808992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:45.809277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:45.809553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.810287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=046cc2b64084423eb14f50f649e7677f) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:45.810599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.817067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770e92-9695-41f2-aa2f-70ec8a0823b8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:45.818678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.833312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 135/267] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:45 2022] GET /identity/v3/role_assignments?user.id=046cc2b64084423eb14f50f649e7677f&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:45.861011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:45.861724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:45.862012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:45.862300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:45.863698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:45.865478 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936f148-d7da-467e-951e-4112dfc9a17c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:45.867608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 133/268] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:45 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:45.873624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:46.021028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec988f-3050-4df7-abaf-449c62ef3ab9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:46.033569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:46.064466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:46.065099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.065399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.065677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.066426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=046cc2b64084423eb14f50f649e7677f) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:46.066746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:46.068311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a91ca-a237-4880-b3ce-a461bcf38a24 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:46.074815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 136/269] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:45 2022] GET /identity/v3/role_assignments?user.id=046cc2b64084423eb14f50f649e7677f&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:46.102086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e4c9-9fc1-4dc1-81d5-04d92a8c9f21 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.102591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e4c9-9fc1-4dc1-81d5-04d92a8c9f21 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.102664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e4c9-9fc1-4dc1-81d5-04d92a8c9f21 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.104042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 134/270] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:46.110499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8d8b-4445-4517-ab8d-492e2b6c3f99 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.110805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8d8b-4445-4517-ab8d-492e2b6c3f99 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.111076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8d8b-4445-4517-ab8d-492e2b6c3f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.137003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-330f8d8b-4445-4517-ab8d-492e2b6c3f99 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:46.138316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-330f8d8b-4445-4517-ab8d-492e2b6c3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:46.235062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 137/271] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:46.385162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b2097-7f0f-4008-a98d-83692fcb44b6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.385571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b2097-7f0f-4008-a98d-83692fcb44b6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.385712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b2097-7f0f-4008-a98d-83692fcb44b6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.387071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 135/272] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:46.392674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.393012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.393247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.421082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:46.422841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:46.442576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6168c4-c418-447b-b5e6-f6b1e9ab11fd None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.442897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6168c4-c418-447b-b5e6-f6b1e9ab11fd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.443300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6168c4-c418-447b-b5e6-f6b1e9ab11fd None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.444992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 136/273] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:46.453061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.453368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.453631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.486750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:46.487999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:46.538243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 138/274] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:46.587770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 137/275] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:46.618149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:46.663011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:46.746576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f13d-3d8f-477f-a254-7d6bb7797b64 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:46.783834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:46.784533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.784819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.785099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.786246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:46.787706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:46.789602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 139/276] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:46.795212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:46.820465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:46.900463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:46.903328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75a1b-b5c0-4636-bb9d-76cebd04369a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:46.953485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:46.954116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:46.954401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:46.954679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:46.955346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:46.956840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.001927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.004734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6924-e47b-45e1-aef2-e53eede09c14 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.050731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 138/277] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:07:46 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:47.051892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.058568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.059417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.059898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.060466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.082098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3aa6c7b5-18a6-4430-ae58-6ac1560bfe3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:47.089439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:47.090678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:47.092294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 140/278] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:47.093282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1426a6-0b95-43f7-9d26-80df1864560e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.093761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1426a6-0b95-43f7-9d26-80df1864560e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.094026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1426a6-0b95-43f7-9d26-80df1864560e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.096113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 141/279] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:47.099062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1426a6-0b95-43f7-9d26-80df1864560e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.191405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.221031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a68f-ca0d-4aee-98e8-1e834e6af8a2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.256974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1426a6-0b95-43f7-9d26-80df1864560e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.264991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.266321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.266724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.267094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.282004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-0ba4cbc1-ce9d-4e34-80d6-41c1d24626fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:47.288453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:47.289630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:47.290762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 139/280] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:47.291795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.291795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.291795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.309460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.310076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.310363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.310638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.311306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:47.311608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:47.313584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:47.313698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.314762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:47.324254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 142/281] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:47.325104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.352131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.409743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 140/282] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:47.410748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.456356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9501c-5dba-487c-8bb0-8b08d35813ba None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.495736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.524854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.526591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.527344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.528100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.529365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:47.530106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:47.533219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.533902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11627f-a4b0-4c3a-afd6-91830735ee19 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.546122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 143/283] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:47.553120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.572508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.573151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.573436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.573714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] PATH_INFO: `/v3/users/a1123d46a1f64c45959128e5db04c843` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.593686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] RBAC: Authorizing `identity:get_user(user_id=a1123d46a1f64c45959128e5db04c843)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:47.595165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.597068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 141/284] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/users/a1123d46a1f64c45959128e5db04c843 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:47.602952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.667687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.686488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe56ee-90bc-4dff-bbe2-94ac878cb344 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.714290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b657d-31c1-4deb-add0-7953261a8124 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:47.728934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.729923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.730389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.730840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] PATH_INFO: `/v3/users/101bdd5543a24f15a91363d45c8f6655` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.732390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] RBAC: Authorizing `identity:get_user(user_id=101bdd5543a24f15a91363d45c8f6655)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:47.735313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.737979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 144/285] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/users/101bdd5543a24f15a91363d45c8f6655 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:47.744656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.744938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:47.754926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:47.755623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:47.755973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:47.756255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:47.757149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, user.id=a1123d46a1f64c45959128e5db04c843, scope.system=all) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:47.757464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:47.759675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:47.767822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 142/286] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/role_assignments?user.id=a1123d46a1f64c45959128e5db04c843&scope.system=all&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:47.827112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:47.952390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de28b3-8b59-44d1-9533-2ade475af056 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.010266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a833-4d8d-46c3-a110-6ba7118e2fbe None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.034026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.034681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.034968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.035251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.036490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:48.038571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:48.049936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 145/287] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:48.057139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:48.075391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.076122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.076457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.076729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.090288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:48.097924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:48.104492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:48.106559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:48.111001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 143/288] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:48.117103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:48.207136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65b95f-0357-4277-bf53-56b6ca549677 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.249353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:48.252415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.253324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.253844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.254334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/101bdd5543a24f15a91363d45c8f6655/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.256613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, user_id=101bdd5543a24f15a91363d45c8f6655, role_id=c868f2c45e5d4238bfd9c1c8cad4d5e5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:48.258245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:48.319075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677e6f-b907-4eb4-9e5b-426bf056ed57 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.356118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.357072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.357552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.358090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.358992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:48.360053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:48.361691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:48.376849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 146/289] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:07:48 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/users/101bdd5543a24f15a91363d45c8f6655/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:07:48.379140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 144/290] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:48.392903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:48.498888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:48.524151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e944f-eb00-4271-a67d-7a9e91366f35 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.563927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.564809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.565165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.565602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] PATH_INFO: `/v3/users/9ec4ac3d03d64456a7da18f064ae4da9` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.567176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=9ec4ac3d03d64456a7da18f064ae4da9)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:48.568682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:48.570339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 147/291] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:48 2022] GET /identity/v3/users/9ec4ac3d03d64456a7da18f064ae4da9 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:48.577592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:48.714715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7435b-c4f1-48b9-a815-3548be4759cf None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:48.730039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:48.755309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:48.756013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:48.756296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:48.756623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:48.854357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:48.856161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:48.992913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 145/292] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:48.999027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:49.121757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801a76-f5f5-4aa7-9e49-2bb2750eb6d0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:49.161280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:49.161912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:49.162194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:49.162464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:49.163586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:49.165041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621ac58-0d2c-4357-9317-b19c7593175e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:49.166728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 148/293] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:48 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:49.173118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:49.187781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:49.275764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f854-f426-4cba-ade1-845c92c3e025 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:49.312714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:49.313330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:49.313610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:49.313880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:49.314605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=9ec4ac3d03d64456a7da18f064ae4da9) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:49.314908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:49.316424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc4585-9a49-4563-83cc-e5d58cb9ecde None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:49.323085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 146/294] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:49 2022] GET /identity/v3/role_assignments?user.id=9ec4ac3d03d64456a7da18f064ae4da9&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:49.922274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af78d3-ac50-4335-b030-5c09e17fdcd6 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:49.922791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af78d3-ac50-4335-b030-5c09e17fdcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:49.922902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af78d3-ac50-4335-b030-5c09e17fdcd6 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:49.924489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 149/295] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:49.930210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add472-51ee-4d67-bf6c-7a9335780652 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:49.930505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add472-51ee-4d67-bf6c-7a9335780652 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:49.930829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add472-51ee-4d67-bf6c-7a9335780652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:49.952400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-33add472-51ee-4d67-bf6c-7a9335780652 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:49.953626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-33add472-51ee-4d67-bf6c-7a9335780652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:50.034750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 147/296] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:50.195909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f99-1811-4694-b86f-8eb908f63572 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.196510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f99-1811-4694-b86f-8eb908f63572 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.197004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f99-1811-4694-b86f-8eb908f63572 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.198811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 150/297] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:50.204590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd19ba-fc03-41e6-8b65-00415d0eec03 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.205112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd19ba-fc03-41e6-8b65-00415d0eec03 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.205584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd19ba-fc03-41e6-8b65-00415d0eec03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.218282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.225557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-66dd19ba-fc03-41e6-8b65-00415d0eec03 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:50.227053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-66dd19ba-fc03-41e6-8b65-00415d0eec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:50.302595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 148/298] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:50.305848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf842f99-1811-4694-b86f-8eb908f63572 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.330742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.422521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf842f99-1811-4694-b86f-8eb908f63572 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:50.457995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:50.458554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.458843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.459122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.466164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622538fd-d89c-4c81-bde1-c9ccc6c60fca None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.466450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622538fd-d89c-4c81-bde1-c9ccc6c60fca None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.466905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622538fd-d89c-4c81-bde1-c9ccc6c60fca None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.469449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 149/299] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:50.472700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:50.474494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.474797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.475071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:50.477818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:50.478704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:50.479916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 151/300] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:50.485431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.501605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:50.502900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:50.571445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.583741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 150/301] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:50.585349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.607117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23179312-5c98-4ce8-8fd4-6619ddd4caaa None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:50.646671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:50.647844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.648260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.648625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.649662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:50.650042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:50.651517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:50.664960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 152/302] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:50.670570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.676870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.714282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a440378-44e6-4f96-a132-c09cf9fffd26 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:50.759683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:50.759683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.759683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.759683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.760106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:50.760106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:50.760106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 151/303] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:50.763972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.782835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d6aa8-d21c-4135-b540-98365f6d9552 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:50.827063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:50.828323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.828624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.828930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] PATH_INFO: `/v3/users/43a43e473dcf4951bd711b73eed433ac` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.838522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.845092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] RBAC: Authorizing `identity:get_user(user_id=43a43e473dcf4951bd711b73eed433ac)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:50.846564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:50.848666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 153/304] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:50 2022] GET /identity/v3/users/43a43e473dcf4951bd711b73eed433ac => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:50.854009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:50.904887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad187-b401-4e12-9390-9d18126b480c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:50.945053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:50.945720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:50.945984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:50.946263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] PATH_INFO: `/v3/groups` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:50.946900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:50.948388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:50.952662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:50.979947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc843cfd-7687-407c-a493-744c5350420c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.015739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 152/305] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Sep 15 19:07:50 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:07:51.015739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.036051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.036790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.037143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.037493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.038316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, user.id=43a43e473dcf4951bd711b73eed433ac, scope.system=all) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:51.038701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:51.040696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1393-438f-4e5c-a263-eaede09a4e9c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:51.054771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 154/306] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 15 19:07:50 2022] GET /identity/v3/role_assignments?user.id=43a43e473dcf4951bd711b73eed433ac&scope.system=all&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:51.126548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b720-ca09-464e-90d6-55fc255b437d None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.127227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b720-ca09-464e-90d6-55fc255b437d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.127720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b720-ca09-464e-90d6-55fc255b437d None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.129427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 155/307] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:51.135376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.136030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.136505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.150952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc133a-4160-4c1a-b7b4-7db5e2296ee6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.168576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:51.169867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:51.215979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.217610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.218482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.219819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.239151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:51.244857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:51.245936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:51.246926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:51.247726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:51.247800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 153/308] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:51.254358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.260327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 156/309] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:51.320542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:51.376778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eabf9-0a51-42b3-87f8-7f3dd158e745 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.417175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.418181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.418667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.419142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.420138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:51.420669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:51.422372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:51.432690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 154/310] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:51.440079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.541463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.545203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b96f2-8f1e-4721-a863-fc2c028c2a80 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.580454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.581101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.581384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.581661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] PATH_INFO: `/v3/users/a1123d46a1f64c45959128e5db04c843` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.582999 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] RBAC: Authorizing `identity:get_user(user_id=a1123d46a1f64c45959128e5db04c843)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:51.584490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:51.590968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 157/311] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/users/a1123d46a1f64c45959128e5db04c843 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:51.592520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.655027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:51.662541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343b540-a171-4c37-8cbb-cb84aead2b7a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.690928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:51.698856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bc97-8ac7-4890-babd-5ebb3e59d770 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.700790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.701665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.702130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.702586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.717730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:51.728407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:51.729955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:51.731434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:51.735917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.736459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 155/312] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:51.738369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.738369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.738369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] PATH_INFO: `/v3/system/users/a1123d46a1f64c45959128e5db04c843/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.738943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a1123d46a1f64c45959128e5db04c843, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:51.743694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:51.748920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:51.795816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 158/313] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Sep 15 19:07:51 2022] PUT /identity/v3/system/users/a1123d46a1f64c45959128e5db04c843/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:07:51.871188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5223b-31e2-4cf8-b52a-133b6b5bf55a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:51.911031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:51.912177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:51.912659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:51.913135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:51.914064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:51.914570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:51.916230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:51.926921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 156/314] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:51.933262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:52.054724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a432b02-522d-44ba-97a7-638579a2dfd8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:52.080668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:52.093429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:52.094606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:52.094893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:52.095172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] PATH_INFO: `/v3/users/101bdd5543a24f15a91363d45c8f6655` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:52.096348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] RBAC: Authorizing `identity:get_user(user_id=101bdd5543a24f15a91363d45c8f6655)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:52.100784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:52.100784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 159/315] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:51 2022] GET /identity/v3/users/101bdd5543a24f15a91363d45c8f6655 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:52.105553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:52.148588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:52.217086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20da98-5adc-4de2-a78c-a19a28914a02 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:52.253100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:52.253906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:52.254189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:52.254461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:52.333704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:52.335087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:52.376893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:52.445824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 157/316] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:52.451553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:52.576679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbe57-58ed-4042-8825-95dbc101c73e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:52.613529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:52.614076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:52.614358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:52.614635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:52.615787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:52.617293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668a42b-b740-4ad2-b81e-d0446714b384 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:52.619095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 160/317] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:52 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:52.625514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:52.735696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb527e-b680-49a7-8201-d7a23fd9e7d9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:52.822914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:52.823539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:52.823849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:52.824127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:52.824884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, scope.project.id=c6bdbb49085042799960d878d3b4a78c, user.id=101bdd5543a24f15a91363d45c8f6655) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:52.825324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:52.826850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c976249-5cfd-4dda-9b52-44279e3c067f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:52.834250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 158/318] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:07:52 2022] GET /identity/v3/role_assignments?user.id=101bdd5543a24f15a91363d45c8f6655&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:53.404693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec80d0-f022-46b7-9528-ddfa788ee4da None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:53.405224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec80d0-f022-46b7-9528-ddfa788ee4da None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:53.405224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec80d0-f022-46b7-9528-ddfa788ee4da None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:53.407567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 161/319] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:53.412979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ad56-24d5-4d1d-8871-1a130554a708 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:53.413414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ad56-24d5-4d1d-8871-1a130554a708 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:53.413552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ad56-24d5-4d1d-8871-1a130554a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:53.420136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:53.433615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a290ad56-24d5-4d1d-8871-1a130554a708 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:53.435334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a290ad56-24d5-4d1d-8871-1a130554a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:53.506238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 159/320] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:53.723514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e062-2387-43e5-a337-640082182b7c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:53.723883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e062-2387-43e5-a337-640082182b7c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:53.724051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e062-2387-43e5-a337-640082182b7c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:53.725509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 162/321] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:53.730749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:53.731043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:53.731323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:53.750634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:53.751875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:53.781263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464e062-2387-43e5-a337-640082182b7c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:53.824638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 160/322] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:53.848477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:53.909299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464e062-2387-43e5-a337-640082182b7c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:53.973699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:53.974403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:53.974756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:53.975096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:53.976281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:53.978056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:53.980814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 163/323] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:07:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:07:53.986220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.060076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:54.100435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e453bc-a384-4e5d-9fea-47bfab1157ce None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.119309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.143907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.144726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.145003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.145295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] PATH_INFO: `/v3/groups` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.145904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:54.147360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550d1e3-e6df-45b1-8a61-c2e5cae3cbb8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:54.245269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 161/324] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Sep 15 19:07:53 2022] POST /identity/v3/groups => generated 226 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:07:54.248764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde696-0fbe-4400-a747-fbd0cdbdf200 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.249425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde696-0fbe-4400-a747-fbd0cdbdf200 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.250743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde696-0fbe-4400-a747-fbd0cdbdf200 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.252511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59fb5-f737-44ea-86f9-37ab1d9bca22 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.253853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 162/325] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:54.262183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.262676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.263164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.286894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:54.297450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.298448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.299525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.299525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.299723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:54.300697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:54.316861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:54.323689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:54.326379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:54.327215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:54.329249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 164/326] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:07:54.342605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.413897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 163/327] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:54.450176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ffa8-e61e-44c2-9859-3d61904a39bf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.492485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.493106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.493389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.493750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.494421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:54.494780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:54.496245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:54.505095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 165/328] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:54.510786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.577292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:54.637805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a80e16-e699-4172-a952-7f36b5406b9b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.677446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.678443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.678934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.679405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] PATH_INFO: `/v3/users/43a43e473dcf4951bd711b73eed433ac` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.680831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] RBAC: Authorizing `identity:get_user(user_id=43a43e473dcf4951bd711b73eed433ac)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:54.682523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:54.684414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 164/329] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:54 2022] GET /identity/v3/users/43a43e473dcf4951bd711b73eed433ac => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:54.690159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.695398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.783366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:54.825146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094acf3-6e1b-4948-81a2-be485ddf130d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.846195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b349f6-9c79-4ad8-b00c-a8c3e54056f9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:54.851698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:54.874793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.875451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.875762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.876041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.896016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:54.900928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:54.901819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:54.902597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 165/330] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:54.909054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:54.909993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:54.910682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:54.910970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:54.911254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] PATH_INFO: `/v3/system/users/43a43e473dcf4951bd711b73eed433ac/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:54.913843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=43a43e473dcf4951bd711b73eed433ac, role_id=c868f2c45e5d4238bfd9c1c8cad4d5e5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:54.915599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:54.982812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 166/331] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Sep 15 19:07:54 2022] PUT /identity/v3/system/users/43a43e473dcf4951bd711b73eed433ac/roles/c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:07:55.043636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6cbf2-22a8-4780-aa24-624e063f25c4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:55.090556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:55.091253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:55.091537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:55.091909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:55.092545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:55.092844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:55.094407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:55.105198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 166/332] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:55.111582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:55.297259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bee1c-34fb-4efe-b732-0ebb6dc56c95 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:55.321876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:55.357129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:55.358183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:55.359146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:55.360042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] PATH_INFO: `/v3/users/a1123d46a1f64c45959128e5db04c843` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:55.362315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=a1123d46a1f64c45959128e5db04c843)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:55.365384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0cbed-d343-4b82-8feb-327f4ddfd5a1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:55.368944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 167/333] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:07:55 2022] GET /identity/v3/users/a1123d46a1f64c45959128e5db04c843 => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:55.376859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:55.431497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:55.505398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f6c1-4ae3-4612-b263-bba8560a19b3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:55.555901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e40b8-0748-4262-b99a-318317154908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:55.556435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e40b8-0748-4262-b99a-318317154908 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:55.556714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e40b8-0748-4262-b99a-318317154908 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:55.556993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e40b8-0748-4262-b99a-318317154908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:55.557775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e40b8-0748-4262-b99a-318317154908 None admin] RBAC: Adding query filter params (role.id=852ee84db43e4ddeba945be2f4d8f1ba, user.id=a1123d46a1f64c45959128e5db04c843, scope.system=all) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:55.558082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e40b8-0748-4262-b99a-318317154908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:55.559520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e40b8-0748-4262-b99a-318317154908 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:55.573461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 167/334] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 15 19:07:55 2022] GET /identity/v3/role_assignments?user.id=a1123d46a1f64c45959128e5db04c843&scope.system=all&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:57.251234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198fc9-59ce-40ad-81ad-6b570323ac11 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:57.251781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198fc9-59ce-40ad-81ad-6b570323ac11 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:57.251857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198fc9-59ce-40ad-81ad-6b570323ac11 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:57.253287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 168/335] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:57.261362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f25e1-cdce-416c-a82e-687cb7e47648 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:57.262432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f25e1-cdce-416c-a82e-687cb7e47648 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:57.262882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f25e1-cdce-416c-a82e-687cb7e47648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:57.288665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-607f25e1-cdce-416c-a82e-687cb7e47648 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:57.290208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-607f25e1-cdce-416c-a82e-687cb7e47648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:57.377085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 168/336] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:57.621765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7049-4f77-4dde-8e44-d620246d44d5 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:57.622468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7049-4f77-4dde-8e44-d620246d44d5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:57.622969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7049-4f77-4dde-8e44-d620246d44d5 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:57.625096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 169/337] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:57.630712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:57.631002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:57.631279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:57.660847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:57.662041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:57.692388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:57.744830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 169/338] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:57.810993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d7049-4f77-4dde-8e44-d620246d44d5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:57.946583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d7049-4f77-4dde-8e44-d620246d44d5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:57.990655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:57.991288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:57.991590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:57.991895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.007574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-caffb85f-c7dc-4c80-98a0-73b4705920b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:58.012623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:58.013552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:58.014355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:58.015535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 170/339] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:58.021410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:58.075314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f81b2-a24e-470d-b9f5-b729e53f6c70 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.075631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f81b2-a24e-470d-b9f5-b729e53f6c70 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.075996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f81b2-a24e-470d-b9f5-b729e53f6c70 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.077515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 171/340] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:07:58.082529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.082870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.083145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.116171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:58.117379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:58.159184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed067dd-f9cf-4e21-a41a-7017b0153b04 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:58.198735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:58.230723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:58.231404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.231709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.233170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.233917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:58.234753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:58.236157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:58.260409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 170/341] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:58.261290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:58.266968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 172/342] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:07:58.267701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:58.320040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:58.490702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526f89f-806e-47b9-81a8-177243b06509 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:58.519442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de11a1-5e3e-4257-88f7-fb8508592e50 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:58.556201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:58.556821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.557107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.557374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.614070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:58.653786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:58.654930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.655361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.656680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.664748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:58.666229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c880df6-bc18-4e27-8072-a5cf4be202ca None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:58.677980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-39360119-8ea7-46ad-af10-97e97a899b06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:58.687879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:58.688840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:58.689612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 171/343] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:58.690752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb755e-6263-41fe-9526-c1ee59b4dacd None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.690846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb755e-6263-41fe-9526-c1ee59b4dacd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.691677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb755e-6263-41fe-9526-c1ee59b4dacd None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.693350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 172/344] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:58.700610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb755e-6263-41fe-9526-c1ee59b4dacd None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:58.790879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 173/345] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:58.792829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.793356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.793843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.824538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:58.825921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:58.880424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:58.907284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb755e-6263-41fe-9526-c1ee59b4dacd None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:58.915408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:58.921087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 174/346] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:58.922530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:58.969509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:58.970136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:58.970417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:58.970691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:58.971376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:58.971721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:58.973238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:58.984943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 173/347] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:58.986364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.114097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c2e4-ac16-496a-a8fd-37b14cbe1139 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.119018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a68ba-c688-4ac9-a197-a143f7714704 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.167344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.168427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.168984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.169356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.170577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:59.172678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:59.173341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.174000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.174285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.174492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 174/348] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:59.174896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.175281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.192125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:59.194608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:59.198634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:59.199574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:07:59.200423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 175/349] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:59.201126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.293162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.310958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60e7e1-5972-4adc-a092-6941e37f103d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.348618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa30ed-6a09-42de-884e-845ed899364b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.355448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.356153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.356459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.356735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.395146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.396722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.396722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.396722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.405882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:59.407386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:59.409121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 176/350] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:59.409953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.450352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:59.451895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b516a-c287-48de-97ff-872bf565f21c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:59.470329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.557285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9637-2037-42d3-b647-db2ff769bb45 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.584371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 175/351] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:07:59.585800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e345-4199-4f22-90c0-0b185e8ed4f7 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.586162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e345-4199-4f22-90c0-0b185e8ed4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.586512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e345-4199-4f22-90c0-0b185e8ed4f7 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.589268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 176/352] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:07:59.589268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853e345-4199-4f22-90c0-0b185e8ed4f7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.603229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.617708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.618343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.618622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.624252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.624252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:59.624252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:59.624252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:59.631556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 177/353] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:59.632145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.745091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853e345-4199-4f22-90c0-0b185e8ed4f7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.765975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.767031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3da-74df-41c8-beaa-bbc7d80d5f3f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:07:59.797158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.797984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.798368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.798654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.813102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:07:59.815577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:07:59.816433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.816867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.817294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:07:59.818173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:07:59.819224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 19:07:59.820236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:07:59.820979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:07:59.821069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 177/354] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:07:59.821069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.821908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:07:59.823705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734ca12-72df-402b-b32e-35ea982e8ff4 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:07:59.830353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 178/355] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:07:59.831929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90808-f98a-408c-823e-b504f2666851 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:07:59.832422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90808-f98a-408c-823e-b504f2666851 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:07:59.832889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90808-f98a-408c-823e-b504f2666851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:07:59.854429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-50b90808-f98a-408c-823e-b504f2666851 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:07:59.856267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-50b90808-f98a-408c-823e-b504f2666851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:07:59.928923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:07:59.963818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 179/356] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:07:59.965148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b90808-f98a-408c-823e-b504f2666851 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:07:59.994656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a85b58-57ea-4099-b95f-11a543cbd332 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.042967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.043532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.043870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.044442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.047285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.053863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.057071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 178/357] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:00.058280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.124119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.143641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b90808-f98a-408c-823e-b504f2666851 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.195176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.197002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.197743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.199856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.200025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f67a4d-a78c-4299-953c-03dc1857ab6a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.249026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.250634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.251136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.251625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.253847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:00.254397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.258431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9423e-9199-4b9b-8a04-015883d148e6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.273549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 179/358] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:00.277090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe480-6fb8-4716-a896-4bb13288498a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.277381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe480-6fb8-4716-a896-4bb13288498a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.277653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe480-6fb8-4716-a896-4bb13288498a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.279225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 180/359] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:00.279851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe480-6fb8-4716-a896-4bb13288498a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.298166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.327849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.330523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.414252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe480-6fb8-4716-a896-4bb13288498a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.416112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.461037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 180/360] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:00.462290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.462401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.466329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.466329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.466329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.466329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.467231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57e1b5-ed96-485e-ad01-a91e14b52c7f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.471216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 181/361] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:00.472870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.473228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.473573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.497693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:00.499186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:00.570493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.595281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 182/362] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:00.596620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.601265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4282aa-87f7-4777-88ce-56d71d569523 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.641713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.642256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.642545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.642820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] PATH_INFO: `/v3/users/43a43e473dcf4951bd711b73eed433ac` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.644267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] RBAC: Authorizing `identity:get_user(user_id=43a43e473dcf4951bd711b73eed433ac)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.646159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.648471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 181/363] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/users/43a43e473dcf4951bd711b73eed433ac => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:00.650450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.724551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.728044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f46526-84f7-4442-aad3-8c44e654db80 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.773472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.774598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.775175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.775851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.791419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:00.796919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:00.797833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:00.798641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:00.801986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 183/364] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:00.808037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.816716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43ea6-a0a8-45ab-b9b3-57d00d31657c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.857169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.857730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.858018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.858299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.859925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.861360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.863378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 182/365] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:00.865577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:00.887533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:00.939226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637519-de54-46f7-923f-e5443858f92e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:00.985604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:00.986844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:00.987335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:00.988200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:00.989142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:00.989659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:00.991479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:00.995122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e5cd-d07d-4baa-a6c5-d79f6b327d1d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.001807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 184/366] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=7647dfe2bb394802a35379f976d57e94 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.003444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.038841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.044823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.045495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.045785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.046066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.047497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] RBAC: Adding query filter params (role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5, user.id=43a43e473dcf4951bd711b73eed433ac, scope.system=all) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:01.047803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.049289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.066652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 183/367] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 15 19:08:00 2022] GET /identity/v3/role_assignments?user.id=43a43e473dcf4951bd711b73eed433ac&scope.system=all&role.id=c868f2c45e5d4238bfd9c1c8cad4d5e5 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.067637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.138731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196fb4-a27a-4bdb-badd-f21d1548d8d4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.182758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.183696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.184689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.185160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.186050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:01.186549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.188273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.198591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 185/368] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.199840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.255413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.308665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.311258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8210-0f73-47b6-b9cd-d383e758c303 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.338808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578968-59b9-41ca-846a-692a4e29f34c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.388304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.389006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.389227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.389501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.391708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.393154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.395760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 184/369] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.396112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.401724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.402361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.402645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.402924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.428856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:01.434333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:01.435235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:01.436097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 186/370] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:01.437055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.532378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d9a3-ebae-4096-823f-dca779e54abd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.563259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983e7c2-6858-4dee-a4eb-12265579548c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.577236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.577885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.578165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.578435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.593502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.604321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f492591-147a-4551-a21b-148adba81eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f492591-147a-4551-a21b-148adba81eae None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f492591-147a-4551-a21b-148adba81eae None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f492591-147a-4551-a21b-148adba81eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f492591-147a-4551-a21b-148adba81eae None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=c6bdbb49085042799960d878d3b4a78c) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f492591-147a-4551-a21b-148adba81eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.613290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f492591-147a-4551-a21b-148adba81eae None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.621649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 187/371] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.623935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f492591-147a-4551-a21b-148adba81eae None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.641167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.683562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.686430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.750511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f492591-147a-4551-a21b-148adba81eae None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:01.798263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:01.799159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:01.799590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:01.800041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:01.801555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:01.802009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:01.806672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:01.818995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 188/372] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:01.823711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 185/373] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:01.825343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.825939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:01.894411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.937736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:01.972721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03811c2-08cd-425b-9427-9dfe4d942f01 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.019147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.019818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.020180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.020564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.038580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84044bd8-96f9-4afc-9374-0c0015c2dd06 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.078684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.079349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.079638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.079950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.083768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.085254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.087488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 186/374] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:02.093693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:02.127815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.129329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.235197 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbc1b2-d11d-4e94-95a0-3a40114153e6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.248088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:02.276521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:02.284336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.285139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.286104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.286403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.287810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.290110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.292602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 187/375] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:02 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:02.299097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:02.317610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 189/376] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:02.323181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:02.449207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13439c-5c7a-4c2e-a684-b1508dfd6661 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.455140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de622fc-3b3d-4b04-8f9d-2b44d9a6242e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.493712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.494436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.494722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.494995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.495872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=c6bdbb49085042799960d878d3b4a78c) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:02.496248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.497784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.498406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.499323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.500333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.500688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] PATH_INFO: `/v3/projects/6238bc86e5b040f1914e478da12747a8` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.501944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] RBAC: Authorizing `identity:get_project(project_id=6238bc86e5b040f1914e478da12747a8)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.503450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.505145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 190/377] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:02 2022] GET /identity/v3/projects/6238bc86e5b040f1914e478da12747a8 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:02.505748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 188/378] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:02 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:02.511077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:02.573919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:02.630818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5aa-57ea-4ced-b8a2-5b7bc075fd25 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.687692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.688742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.689238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.689740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] PATH_INFO: `/v3/groups/77a9130d62fc469481d67e8042bee5bd` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.700485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] RBAC: Authorizing `identity:get_group(group_id=77a9130d62fc469481d67e8042bee5bd)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.703097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8027a4-8bcf-41f8-83b5-06bd2a65e049 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.705274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 191/379] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:02 2022] GET /identity/v3/groups/77a9130d62fc469481d67e8042bee5bd => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:02.711666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:02.773479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda98cf5-99ef-44d6-b6d7-630ccd0f3bcf None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.774128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda98cf5-99ef-44d6-b6d7-630ccd0f3bcf None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.774626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda98cf5-99ef-44d6-b6d7-630ccd0f3bcf None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.776577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 192/380] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:02.786324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.786987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.787473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.789524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:02.814516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:02.816365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:02.874460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c91a-6ba4-49df-a2a6-ff66e993f7e6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:02.911892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:02.912511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:02.912795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:02.913065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:02.913799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] RBAC: Adding query filter params (group.id=77a9130d62fc469481d67e8042bee5bd, role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=6238bc86e5b040f1914e478da12747a8) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:02.914109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:02.915559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:02.922717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 189/381] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:02 2022] GET /identity/v3/role_assignments?group.id=77a9130d62fc469481d67e8042bee5bd&scope.project.id=6238bc86e5b040f1914e478da12747a8&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:02.961612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 193/382] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:03.338893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:03.483351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810505-3253-4991-a583-f1e6b031024c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:03.492134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:03.530297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:03.531798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:03.532081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:03.532357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:03.547700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:03.552716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:03.553655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:03.554900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 190/383] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:03.559883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:03.608222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:03.705067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b15ca-e0b3-4d50-a7c9-1aadff7f9635 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:03.758759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:03.759379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:03.759687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:03.759965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:03.760649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:03.760959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:03.762366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:03.775872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 194/384] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:08:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:03.781984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:03.940842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0e31-4bdb-4e2a-ae7d-4862dc5273b6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:03.995203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:03.995877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:03.996160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:03.996433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.000797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:04.002325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0cd94-9d66-4273-90e5-0ef41ec59859 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:04.004433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 191/385] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:03 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:04.010009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:04.148496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff4dd2-6a2d-4270-bf87-d5588f91979c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.148842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff4dd2-6a2d-4270-bf87-d5588f91979c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.149053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff4dd2-6a2d-4270-bf87-d5588f91979c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.150593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 192/386] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:04.165434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.165750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.166059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.168651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:04.174684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b97742-770a-41fd-a6a3-0583937d2c69 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:04.198418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:04.199557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:04.213906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:04.214556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.214865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.215219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.216494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:04.218348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:04.220461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 195/387] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:04 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:04.232700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:04.236195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:04.283160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 193/388] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:04.480864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da766e8-7e94-4417-b4c5-94abd09a9acd None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:04.550116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:04.551533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.552252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.552960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.559049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, group_id=e8c9ce0d502c4d8da2a779f8df6ab86a, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:04.562069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8232-de10-4623-bcd9-227f796db29e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:04.643160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 196/389] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 15 19:08:04 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:04.791920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:04.794831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876d0b-586f-492c-8372-1b84ef3f50b6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.795414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876d0b-586f-492c-8372-1b84ef3f50b6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.796144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876d0b-586f-492c-8372-1b84ef3f50b6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.798168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 197/390] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:04.804521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:04.805357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:04.806111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:04.842505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:04.845645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:04.904155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:04.980485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 198/391] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:04.989614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd76dba-da60-4685-a80f-46e600606f27 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.024765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.025408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.025689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.025961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.037718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:05.042875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:05.043820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:05.044621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 194/392] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:05.050808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.174101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fff0-0254-47ce-a328-91b0af986f0e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.222420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.223046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.223389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.223692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.224367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:05.224668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:05.226043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:05.235158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 199/393] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:05.240603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.275737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:05.287230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.324149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:05.370124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d537-daa7-49db-b4aa-530fb89bac80 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.411730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.412428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.412709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.412984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.414130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:05.415795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:05.417980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 195/394] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:05.429351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.485754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875f159-cdf2-4ac7-aeed-8f980f3c3b5a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.530714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.531368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.531677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.531959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.545713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b3eda9be-9208-42c4-bea0-60a892d1b100 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:05.553265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:05.554301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:05.555459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 200/395] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:05.556170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8a26-6d83-41c3-8619-965f5892d5b7 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.559947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8a26-6d83-41c3-8619-965f5892d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.560534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8a26-6d83-41c3-8619-965f5892d5b7 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.563370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 201/396] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:05.568038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8a26-6d83-41c3-8619-965f5892d5b7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.605662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:05.617882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b8b4e-17f5-484c-bbbc-674ec00687e8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.662103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:05.664140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.664710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.664993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.665268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.666922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:05.668616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcc0bf-8b1a-439b-8a52-5e6630b484f3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:05.670789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 196/397] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:05.673048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.673560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.674036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.696819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:05.697011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8a26-6d83-41c3-8619-965f5892d5b7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.698837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:05.737704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.738295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.738587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.738865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.739539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:05.739893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:05.741513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:05.755851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 202/398] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:05.755851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.793676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 197/399] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:05.795180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:05.904872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:05.929361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828302b2-6d23-4739-8f3f-dd46712d17e1 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.941166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa669e-c627-4b10-a112-b02b9e9a4084 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:05.971743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:05.972741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:05.973247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:05.973742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:05.975155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:05.977000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:05.978834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 198/400] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:05.985020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.032853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.033947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.035431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.038768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.042285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] RBAC: Authorizing `identity:create_grant(project_id=fb57dff1ef814c0992dc5e148d1ae3f3, group_id=e8c9ce0d502c4d8da2a779f8df6ab86a, role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:06.044235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2087d-a298-4f0c-8c08-11a257ba5efa None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:06.126127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:06.132906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 203/401] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 15 19:08:05 2022] PUT /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/7647dfe2bb394802a35379f976d57e94 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:06.135049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fc7-3626-4cf1-b8e5-676fb5e80419 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.136118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fc7-3626-4cf1-b8e5-676fb5e80419 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.136794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fc7-3626-4cf1-b8e5-676fb5e80419 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.139641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 204/402] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:06.145909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.146890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.147931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.154778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228fc59-85a7-4c91-a3be-bed3e68dc9c9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.174780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:06.176752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:06.205297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.205944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.206226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.206502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.207685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:06.210165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:06.212306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 199/403] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:05 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:06.213241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.249075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:06.268531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 205/404] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:06.269202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.341023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27704-2e6f-43b9-bac5-e1c07f684fcc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.382646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.383278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.383556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.383859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.397063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:06.403338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:06.404289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:06.405102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 200/405] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:06.411067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.428270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399700a2-86e4-4550-aa65-0fc6b318b8f5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.467261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:06.478316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.482920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.482920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.482920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.482920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, group_id=e8c9ce0d502c4d8da2a779f8df6ab86a, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:06.484096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:06.519341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:06.535790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273af83-37df-4aac-b956-5fd38a8081cc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.572703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.573310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.573592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.573920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.574636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:06.574994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:06.576426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:06.587450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 206/406] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 15 19:08:06 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:06.593504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 201/407] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:06.599755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.623070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.807055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf637e77-3c96-453e-b445-0abff5f5a52b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.827736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e48c0-f69d-40f3-a90a-30f0f445cbbf None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:06.848175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.849248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.849820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.849993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.866462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:06.872306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:06.873282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:06.874736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 202/408] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:06.878538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:06.879294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:06.879483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:06.879792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:06.880844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.880951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:06.882399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:06.884822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 207/409] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:06.892728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:06.899148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:07.019694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d48c0d-4645-4545-941a-633ecee7f11e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.028883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:07.060940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.062529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.063019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.063513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.064397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:07.068153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.068153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.080945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 203/410] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:07.082676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:07.119520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccae3be-5e6a-4bdb-b777-d016b237478c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.160113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:07.193217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca91d-d0fa-4eaa-826b-b0f9803009f1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.217702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.218898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.219179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.219451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.224612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.226145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.228345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 208/411] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:06 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:07.237115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:07.242650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.243723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.244355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.244835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] PATH_INFO: `/v3/groups/77a9130d62fc469481d67e8042bee5bd` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.246248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] RBAC: Authorizing `identity:get_group(group_id=77a9130d62fc469481d67e8042bee5bd)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.248034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.254818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 204/412] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:07 2022] GET /identity/v3/groups/77a9130d62fc469481d67e8042bee5bd => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:07.256452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:07.365208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3611-0e98-40d9-92b3-cfda6845922d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.401931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.402979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.403470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.403979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] PATH_INFO: `/v3/projects/6238bc86e5b040f1914e478da12747a8` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.405849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] RBAC: Authorizing `identity:get_project(project_id=6238bc86e5b040f1914e478da12747a8)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.407504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1649d01-5b9d-458b-87d8-86afac57c776 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.415838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 205/413] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:07 2022] GET /identity/v3/projects/6238bc86e5b040f1914e478da12747a8 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:07.417600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394af22-6cc4-4102-aee8-4cf1bd8b9d7a None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.418088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394af22-6cc4-4102-aee8-4cf1bd8b9d7a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.418577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394af22-6cc4-4102-aee8-4cf1bd8b9d7a None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.420127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 206/414] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:07.421885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394af22-6cc4-4102-aee8-4cf1bd8b9d7a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:07.431027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bdbea-1bc8-41b2-a9fd-2a98abd7278c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.448052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:07.463947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:07.479879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.480553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.480832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.481105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.483304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] RBAC: Authorizing `identity:create_grant(project_id=c6bdbb49085042799960d878d3b4a78c, group_id=e8c9ce0d502c4d8da2a779f8df6ab86a, role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.486362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca5104-3867-40be-be22-4191ed567ead None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.543923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 209/415] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 15 19:08:07 2022] PUT /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c/groups/e8c9ce0d502c4d8da2a779f8df6ab86a/roles/7647dfe2bb394802a35379f976d57e94 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:08:07.545023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394af22-6cc4-4102-aee8-4cf1bd8b9d7a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:07.545477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.545765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.546843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.587067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:07.592024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:07.617393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:07.620277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:07.620854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:07.621425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] PATH_INFO: `/v3/projects/6238bc86e5b040f1914e478da12747a8/groups/77a9130d62fc469481d67e8042bee5bd/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:07.625563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] RBAC: Authorizing `identity:create_grant(project_id=6238bc86e5b040f1914e478da12747a8, group_id=77a9130d62fc469481d67e8042bee5bd, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:07.627378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:07.685480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 207/416] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 15 19:08:07 2022] PUT /identity/v3/projects/6238bc86e5b040f1914e478da12747a8/groups/77a9130d62fc469481d67e8042bee5bd/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:08:07.714171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 210/417] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:08.027776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:08.114107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:08.152206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c9d3e-359b-47ae-a41f-7f98293fe489 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:08.190941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:08.191596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:08.191911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:08.192189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:08.204047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:08.209375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:08.210288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:08.211094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 208/418] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:08.216425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:08.377334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022705e-1922-464b-9e6d-1233dfb26792 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:08.403333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:08.426050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:08.426678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:08.426961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:08.427236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:08.427954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:08.428260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:08.429631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:08.446011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 211/419] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:08.451933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:08.572362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd4673-52c7-40d5-8b72-df04fc782377 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:08.598633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:08.607536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:08.608180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:08.608966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:08.609246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:08.680590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:08.682061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:08.790886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 209/420] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:08.796751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:08.929790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239c0ff-2bf3-4fae-b787-458ccf973cfa None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:08.981290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:08.981936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:08.982220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:08.982495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:08.984234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:08.985736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a0c80-2fd8-4668-b165-d45fa0443aff None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:08.988546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 212/421] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:08 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:08.992993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:09.001078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9f2-8077-436c-8a40-5ea24ee0859e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.001434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9f2-8077-436c-8a40-5ea24ee0859e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.001720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9f2-8077-436c-8a40-5ea24ee0859e None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.003302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 213/422] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:09.013717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.014173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.014456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.043231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:09.044360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:09.056488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:09.131972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:09.183389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065286-3c40-4585-a467-afa9d0d23a1c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:09.205310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 214/423] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:09.269700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:09.270811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.271302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.271828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.273750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:09.276604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b96c02-32bc-489a-87ad-d598248b4e28 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:09.280047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 210/424] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:08 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:09.286243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:09.434639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4118698-0730-4714-ab22-3b8bc070961e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:09.472069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:09.472695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.472978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.473255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.474125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:09.474367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:09.475958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:09.477963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be05527-dc74-4df8-8bb5-0c681520b688 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.478247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be05527-dc74-4df8-8bb5-0c681520b688 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.478520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be05527-dc74-4df8-8bb5-0c681520b688 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.480059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 211/425] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:09.486320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 215/426] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:09 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:09.494034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.494345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.494646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.514025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:09.515841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:09.549426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:09.644906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:09.655904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 212/427] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:09.730843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67991d7-f9e1-48bb-8a1d-54749e47ed37 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:09.767455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:09.768120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.768405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.768670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.781478 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:09.791530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:09.792528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:09.793319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:09.793934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 216/428] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:09.801170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:09.828673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:09.931060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34346d53-2d67-4945-b849-2087cde95cbd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:09.973138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:09.991347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:09.992026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:09.992317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:09.992597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:09.995687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:09.995687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:09.995687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.005461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 213/429] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:10.012444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.135564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.140188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd3ead-6443-4ce8-b94c-6ea292ebde12 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.152375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.172507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07b23-f027-4c9a-85ff-e040db7a5980 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.184293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.184989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.185377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.186017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.209091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-cdc7b976-b964-4ca9-a652-d07462d91f43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:10.216008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:10.216972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 19:08:10.218338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:10.224431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 217/430] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:10.226822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214755b3-5c74-4faf-a3d1-4aff47ae0542 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.227212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214755b3-5c74-4faf-a3d1-4aff47ae0542 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.227504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214755b3-5c74-4faf-a3d1-4aff47ae0542 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.232530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 218/431] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:10.232530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214755b3-5c74-4faf-a3d1-4aff47ae0542 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.234211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.234850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.235129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.235395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.324923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:10.326629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.371477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214755b3-5c74-4faf-a3d1-4aff47ae0542 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.413747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.414352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.414633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.414905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.415564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] RBAC: Adding query filter params (name=member) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:10.415889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:10.418012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edcf90-4093-414a-986d-9401e47d8685 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.423451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.426694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 219/432] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:10.432674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.432674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.432674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.447567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 214/433] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:10.447567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.452305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:10.453605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:10.469186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.527753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 220/434] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:10.532506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.558905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27612c3c-4a8e-47a8-8fd6-24e4731f64e2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.596525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.597568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.598022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.598462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.663523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.671032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9340d95-ee1b-4df9-aab7-1212b44dedba None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.677066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:10.678632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e6c7d-4ee2-4355-8d13-f2b3e9579052 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.706089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.706892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.707170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.707448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] PATH_INFO: `/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.708606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] RBAC: Authorizing `identity:get_project(project_id=fb57dff1ef814c0992dc5e148d1ae3f3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:10.710037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee8bd1-8342-4989-b9ed-39c3680d8a06 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.711637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 221/435] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/projects/fb57dff1ef814c0992dc5e148d1ae3f3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:10.717236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62d887-2958-4d33-bc06-4dd37ced8b4e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.717518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62d887-2958-4d33-bc06-4dd37ced8b4e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.717848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62d887-2958-4d33-bc06-4dd37ced8b4e None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.719568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 222/436] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:10.720279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62d887-2958-4d33-bc06-4dd37ced8b4e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.745259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.789566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 215/437] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:10.790828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.791118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.791405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.809440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:10.810639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:10.833122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62d887-2958-4d33-bc06-4dd37ced8b4e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.866719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:10.867269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:10.867549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:10.867854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:10.868971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:10.870397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:10.871977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 223/438] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:10.872729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.884900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 216/439] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:10.885752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:10.903178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.972629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:10.988429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133ecfb-55fd-41db-b2ce-719fd3bcae46 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:10.996823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbb7af-8299-43ad-a1f4-c55a35189af0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.025157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.025788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.026190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.026467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.027717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.029513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.032058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.032691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.032979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.033259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.037240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 224/440] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.037240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.046304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:11.051266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:11.052242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:11.053053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 217/441] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:11.053723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.143967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.149811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545359da-dbc3-41b0-beac-1bf731d40ea5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.166559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c7b1a-8fe6-42f0-a2ca-5518567f128f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.183734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.184353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.184645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.184928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.185660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:11.185974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.187996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.193872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 225/442] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=fb57dff1ef814c0992dc5e148d1ae3f3&role.id=7647dfe2bb394802a35379f976d57e94 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.194655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.205370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.206293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.206770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.207227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.208735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.210493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.212424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 218/443] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.213434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.223222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.318571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523b95-c444-46da-b66f-57b24383c1a8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.337839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae8f4-55fd-441b-925b-2b4d0723fba9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.357075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.357693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.357973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.358245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.359492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:11.359830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.361227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.371081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 226/444] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.371962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.380859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.381505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.381789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.382064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.393020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:11.397931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:11.398792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:11.399870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 219/445] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:11.400825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.408838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.474543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.502975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565187-cff6-49f4-b52f-704fdd75f9b1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.520170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a974f28-dfa3-41c6-8acd-4b4e027dafbe None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.543748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.544363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.544651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.544930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.545663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=852ee84db43e4ddeba945be2f4d8f1ba, scope.project.id=c6bdbb49085042799960d878d3b4a78c) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:11.546573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.548030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.554119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 227/446] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=852ee84db43e4ddeba945be2f4d8f1ba => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.554941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.561385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.562639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.562930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.563206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.654801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.657113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.660351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.692109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15bd49-a3f4-4b79-8584-6b7d77caa5a8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.737503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cb953-768c-4930-a195-f666373b3922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.738124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cb953-768c-4930-a195-f666373b3922 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.738405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cb953-768c-4930-a195-f666373b3922 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.738680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cb953-768c-4930-a195-f666373b3922 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.739360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cb953-768c-4930-a195-f666373b3922 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:11.739767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cb953-768c-4930-a195-f666373b3922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.741175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cb953-768c-4930-a195-f666373b3922 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.751151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 228/447] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:11.757535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cb953-768c-4930-a195-f666373b3922 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.767778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.788606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 220/448] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:11.794656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:11.888920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cb953-768c-4930-a195-f666373b3922 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.925399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fdeeb-6a6d-45f4-9392-7e49e1deb137 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:11.931671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:11.935582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.936500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.936799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.937826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.981326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:11.983458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:11.983458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:11.984306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] PATH_INFO: `/v3/projects/c6bdbb49085042799960d878d3b4a78c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:11.985514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] RBAC: Authorizing `identity:get_project(project_id=c6bdbb49085042799960d878d3b4a78c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:11.986948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:11.988909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 221/449] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/projects/c6bdbb49085042799960d878d3b4a78c => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:11.994254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:12.016947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.017824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.027132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:12.103574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9981-ab0c-4071-a76a-34a3e78fa966 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:12.124866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 229/450] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:12.130520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:12.140676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:12.141543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:12.141824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:12.142107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] PATH_INFO: `/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:12.143232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] RBAC: Authorizing `identity:get_group(group_id=e8c9ce0d502c4d8da2a779f8df6ab86a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.144708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.146275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 222/451] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:11 2022] GET /identity/v3/groups/e8c9ce0d502c4d8da2a779f8df6ab86a => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:12.152412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:12.199560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:12.236971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3474f-136f-4f6e-9586-71060b293925 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:12.244123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:12.258564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79562-b456-473f-a1a5-355c912a645b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:12.280140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:12.281315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:12.281591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:12.281857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] PATH_INFO: `/v3/projects/6238bc86e5b040f1914e478da12747a8` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:12.283187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] RBAC: Authorizing `identity:get_project(project_id=6238bc86e5b040f1914e478da12747a8)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.284428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.286026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 230/452] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:08:12 2022] GET /identity/v3/projects/6238bc86e5b040f1914e478da12747a8 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:12.291509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:12.293436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:12.294037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:12.294323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:12.294596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:12.295316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] RBAC: Adding query filter params (group.id=e8c9ce0d502c4d8da2a779f8df6ab86a, role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=c6bdbb49085042799960d878d3b4a78c) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:12.295629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.297060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.302756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 223/453] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:12 2022] GET /identity/v3/role_assignments?group.id=e8c9ce0d502c4d8da2a779f8df6ab86a&scope.project.id=c6bdbb49085042799960d878d3b4a78c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:12.402500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8755f-5869-4a24-a71e-f51bba182181 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:12.438572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:12.439230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:12.439513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:12.439824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] PATH_INFO: `/v3/groups/77a9130d62fc469481d67e8042bee5bd` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:12.441054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] RBAC: Authorizing `identity:get_group(group_id=77a9130d62fc469481d67e8042bee5bd)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.442528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9f9e7-6d7e-47d8-9737-48677c6bf207 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.444359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 231/454] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 15 19:08:12 2022] GET /identity/v3/groups/77a9130d62fc469481d67e8042bee5bd => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:12.450478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:12.560083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856974-e3d3-4e16-9371-f18ab26c1e56 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:12.562409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:12.594713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:12.595569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:12.596109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:12.596552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:12.597459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] RBAC: Adding query filter params (group.id=77a9130d62fc469481d67e8042bee5bd, role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=6238bc86e5b040f1914e478da12747a8) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:12.598051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:12.599813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50cfe3-bdb8-4f12-a0c8-25cc0b79c89d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:12.606216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 224/455] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 15 19:08:12 2022] GET /identity/v3/role_assignments?group.id=77a9130d62fc469481d67e8042bee5bd&scope.project.id=6238bc86e5b040f1914e478da12747a8&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:15.533287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d220-b4b8-438e-b2c1-c35e681f73f6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.533820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d220-b4b8-438e-b2c1-c35e681f73f6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.533820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d220-b4b8-438e-b2c1-c35e681f73f6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.535467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 232/456] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:15.540759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a6e7e-270e-49ac-898c-5e40f088a333 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.541181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a6e7e-270e-49ac-898c-5e40f088a333 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.541341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a6e7e-270e-49ac-898c-5e40f088a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.565398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a83a6e7e-270e-49ac-898c-5e40f088a333 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:15.566741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a83a6e7e-270e-49ac-898c-5e40f088a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:15.597496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e01241-d2ee-44e4-b7b8-1aa30d619822 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.598078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e01241-d2ee-44e4-b7b8-1aa30d619822 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.598534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e01241-d2ee-44e4-b7b8-1aa30d619822 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.600479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 233/457] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:15.605943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.606505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.606989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.639797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:15.642161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:15.688550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 225/458] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:15.712729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0255e-be18-4917-97d1-a9f144de4c89 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.713355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0255e-be18-4917-97d1-a9f144de4c89 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.713830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0255e-be18-4917-97d1-a9f144de4c89 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.715361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 226/459] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:15.721019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:15.721538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:15.722057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:15.726227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:15.751253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:15.753653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:15.790099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 234/460] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:15.856129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 227/461] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:16.087926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.101581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.165451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.201620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfda57-75fd-42b4-8141-c4f36814587b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.239343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.240594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.241214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.241720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.256088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.258158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.258661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc62c-079f-47eb-87ff-7c5314a2994f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:16.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:16.278552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:16.279407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:16.279407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.279407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:16.279407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 235/462] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:16.279842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.303080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.303780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.304067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.304344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.310305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.315264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.316748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-40751358-e1bd-4bc4-a7ac-5bee5303167d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:16.332483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:16.333442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:16.334905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:16.334905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.334905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:16.334905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 228/463] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:16.335132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0df3-351a-4c56-b67d-f142cfc1cc55 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.335387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0df3-351a-4c56-b67d-f142cfc1cc55 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.335681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0df3-351a-4c56-b67d-f142cfc1cc55 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.339832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 229/464] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:16.339832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e0df3-351a-4c56-b67d-f142cfc1cc55 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.400428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80414886-aa2e-4327-9dc2-d31aa440cb8b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.431215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.441178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.441991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.442104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.442372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.454721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.456554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.457175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e0df3-351a-4c56-b67d-f142cfc1cc55 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:16.474074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:16.475060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:16.475938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:16.475938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:16.475938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:16.475938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 236/465] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:16.476176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.503015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.503633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.503968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.504327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.504977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:16.505281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.506764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c4e68-0b18-4ce1-81de-80ee004154c7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.519764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 230/466] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:16.525671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.525969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.526271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.545302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:16.547094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:16.559674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.582237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7628a6-2f34-4330-ba4b-1adf17b119c0 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.613237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 231/467] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:16.614293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.625371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.625989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.626385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.626665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.627323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:16.627642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.629066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.646872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 237/468] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:16.646872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.693132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.724296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34f5c8-0f1f-4946-b0fe-ac16657c97e2 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.755027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11144950-fa5c-4cf0-90df-7aa91dd3b6d4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.761024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.761636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.761915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.762186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.762828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:16.763128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.765083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8a2d6-0fa3-48e3-af25-ec57678770e1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 232/469] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d176-45cc-4257-8e12-283ae348e3eb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d176-45cc-4257-8e12-283ae348e3eb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d176-45cc-4257-8e12-283ae348e3eb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 233/470] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:16.782004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53d176-45cc-4257-8e12-283ae348e3eb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.802153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.802768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.803054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.803793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.804456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.806422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:16.818706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.883159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53d176-45cc-4257-8e12-283ae348e3eb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:16.890617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 238/471] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:16 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:16.891312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:16.919179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:16.926409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:16.927243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:16.927402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:16.927688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:16.928373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:16.929877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b05618-fe6c-48e3-8e86-c78712b6f2a2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.014080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d29e-9615-490a-bc59-f1a8391d7180 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.016772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 234/472] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:16 2022] POST /identity/v3/users => generated 246 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:17.019220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.019543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.019817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.044692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:17.046057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:17.070728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.071751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.072914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.073429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.074550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:17.076240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.144773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 235/473] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:17.145549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:17.194932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 239/474] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:16 2022] POST /identity/v3/users => generated 248 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:17.284318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0462ce-c0f2-4d74-81d4-f589296c4d50 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.304628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:17.330263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.330934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.331224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.331503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.349561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:17.351038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:17.373599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:17.380047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:17.380940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:17.380940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:17.380940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.380940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 236/475] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:17.381607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:17.454913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:17.464280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:17.513620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484bcf3-7357-4e57-81a4-d8a22ececa84 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.551467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.552183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.552471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.552748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.553413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:17.553784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:17.555184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.568917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 240/476] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:17.576739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:17.603004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc777-28ea-4e3a-b37e-9cd92e8413b3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.672078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.673569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.674218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.675304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.676177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:17.679063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.729880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97971c-b203-47e9-95a6-e32ab5af0834 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.741687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.sql.core [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 19:08:17.741687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 19:08:17.741687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [parameters: {'id': 'd012378e014f474f92123af88e2491cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 19:08:17.741687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81559) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 19:08:17.756663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result.read() Sep 15 19:08:17.765156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:08:17.766212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application yield resource Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise self.value Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application session.commit() Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 19:08:17.767354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.session.flush() Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application compat.raise_( Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 19:08:17.768457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:08:17.769336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result.read() Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:08:17.770128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [parameters: {'id': 'd012378e014f474f92123af88e2491cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:17.770907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 19:08:17.771730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.777534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 237/477] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 15 19:08:17 2022] POST /identity/v3/roles => generated 144 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:17.789830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:17.825542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.826007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.826291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.826621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.827323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:17.828837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:17.834604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:17.912905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559687d-d2ff-432d-80de-db9e73ad5314 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:17.957600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:17.958535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:17.958807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:17.959090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:17.970544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-53a253a4-63b4-4f4f-8ea8-ea4ed28b2274 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:17.976292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:17.977217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 19:08:17.978037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:17.978637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 241/478] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:17 2022] POST /identity/v3/users => generated 249 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:08:17.978707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 238/479] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:08:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:17.983858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:18.130224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2de01-a686-4d02-8eea-09a2f50a8a52 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:18.187057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:18.188100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:18.188591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:18.189090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:18.190041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:18.190578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:18.192288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c04a-bf23-4479-801e-57cc842a2f06 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:18.202329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 242/480] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:19.654660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963dcc9-8d93-4e79-8306-057a86495ffd None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.655987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963dcc9-8d93-4e79-8306-057a86495ffd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.656476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963dcc9-8d93-4e79-8306-057a86495ffd None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:19.658269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 239/481] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:19.664363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162d7b-4550-4902-8301-9e7b275d658a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.665350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162d7b-4550-4902-8301-9e7b275d658a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.665828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162d7b-4550-4902-8301-9e7b275d658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:19.692851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7b162d7b-4550-4902-8301-9e7b275d658a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:19.694445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7b162d7b-4550-4902-8301-9e7b275d658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:19.718321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:19.767060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 243/482] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:19.792368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2706-52d6-4b19-b03a-56ed604c2ecb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.792677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2706-52d6-4b19-b03a-56ed604c2ecb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.792928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2706-52d6-4b19-b03a-56ed604c2ecb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:19.794258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 240/483] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:19.799865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d872-7317-4672-9116-33e5947b90bc None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.800140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d872-7317-4672-9116-33e5947b90bc None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.800419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d872-7317-4672-9116-33e5947b90bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:19.818999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ce07d872-7317-4672-9116-33e5947b90bc None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:19.820218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ce07d872-7317-4672-9116-33e5947b90bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:19.896498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 244/484] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:19.987441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cbff1-f8b9-4fc1-89db-8ebb636b9a8b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.987786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cbff1-f8b9-4fc1-89db-8ebb636b9a8b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.988027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cbff1-f8b9-4fc1-89db-8ebb636b9a8b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:19.989679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 241/485] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:19.994661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:19.994958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:19.995233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.015100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:20.016321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:20.097146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 245/486] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:20.219230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cbff1-f8b9-4fc1-89db-8ebb636b9a8b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.276380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.301524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:20.401767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:20.409962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9c28d-52da-4746-a9ad-f27c48e9f5b0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.444430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cbff1-f8b9-4fc1-89db-8ebb636b9a8b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.446683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.447719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.447968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.448353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.460935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:20.466289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:20.467381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:20.468785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 246/487] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:20.469478 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.494206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.494939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.495227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.495507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.511106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:20.517246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:20.519402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 242/488] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:20.521485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.560773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:20.586255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e853a-cd45-40dd-a807-30af9869723e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.626741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.627555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.628164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.628568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.642195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:20.647785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:20.648942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:20.649946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:20.652910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 247/489] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:20.653105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.661953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d9ee-0617-4822-9709-8b0dbcd823ff None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.707510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.708216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.708686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.708970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.709963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:20.710687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:20.712106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:20.727861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 243/490] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:20.728699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.751024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:20.771949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ace8d-e3ea-4497-8125-b11aa9035f33 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.811788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.812793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.813219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.813658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.814348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:20.814747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:20.817489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:20.821330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:20.833186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 248/491] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:20.833186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.853304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2707-f610-4b7a-b594-93237f949cab None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.891172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.891817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.892105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.892423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.893050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:20.893507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:20.894718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7194dc9-d3bb-4285-a173-cba8dd777503 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:20.904683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 244/492] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:20.904683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53207d-cf4f-4c27-9304-1be1e82afedb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.905718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53207d-cf4f-4c27-9304-1be1e82afedb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.905963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53207d-cf4f-4c27-9304-1be1e82afedb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:20.911502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 245/493] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:20.911502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53207d-cf4f-4c27-9304-1be1e82afedb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:20.951498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc11d3-7997-43ab-af54-74e5439be476 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:20.990627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:20.991557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:20.992069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:20.992519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.009592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.010130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.011851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:21.037866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.039681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:21.041392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:21.041392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.041392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:21.041392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 249/494] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.041392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.052593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53207d-cf4f-4c27-9304-1be1e82afedb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.091137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.091795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.092080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.092358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.104620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.106062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.107021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9742d192-d774-4751-abe8-a8d49cf9db49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:21.121874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.122827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:21.123791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:21.123791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.123791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:21.123791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 246/495] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.124865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.124960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.125239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.144876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:21.146756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:21.165092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e1bf-d211-4a34-a938-212d0410aabb None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.201941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.202563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.202842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.203114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.213424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.214856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.215987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 247/496] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:21.216828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:21.229051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.230045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:21.230905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:21.230905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.230905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:21.230905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 250/497] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.230905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.292448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.337491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab4e1-fdc6-4a41-85ed-d34dd21cebbe None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.337719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7722-646c-4b9f-afd1-28a798eb90f3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.339262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.371873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.372480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.372759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.373032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.374303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:21.374402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.374605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.374979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.375257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.375528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.376017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.376202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:21.376510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.377946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b346c-7315-48fd-8ca9-bd6ea52de1c2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.394400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 248/498] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:21.394400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886a68-ed53-4960-be5a-196f71eae342 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.394400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886a68-ed53-4960-be5a-196f71eae342 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.394400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886a68-ed53-4960-be5a-196f71eae342 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.394400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 249/499] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:21.394822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886a68-ed53-4960-be5a-196f71eae342 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.395585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 251/500] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:21.395585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.502256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49fa3-3e31-4f2f-b8ec-c0b6ca238454 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.504419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886a68-ed53-4960-be5a-196f71eae342 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.522532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.539508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.539770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564570ce-be82-4931-be32-994f49689f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.540223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.540387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564570ce-be82-4931-be32-994f49689f17 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.540512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.540667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564570ce-be82-4931-be32-994f49689f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.540798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.540940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564570ce-be82-4931-be32-994f49689f17 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.541396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:21.541700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.550277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.550483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564570ce-be82-4931-be32-994f49689f17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.550483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564570ce-be82-4931-be32-994f49689f17 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.555667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 250/501] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:21.556638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-564570ce-be82-4931-be32-994f49689f17 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:21.563536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.564631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:21.565558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 252/502] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.566192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.566779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.566779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.587784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:21.589009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:21.606421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.657630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 253/503] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:21.658488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.661437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf20a-4eb0-4771-bbe0-0547496483e1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.698950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.699617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.699877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.700152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.708034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.709483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:21.723429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.724681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 251/504] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.725839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.771382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:21.848774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0a1d4-cf45-447f-b7de-f2d39a3ecab1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.849306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a90b9-8873-4ea1-b8b6-2a309a51b584 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:21.883737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.884376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.884662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.884938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.890445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:21.891199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:21.891475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:21.891784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:21.896924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.899944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.900226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:21.901716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:21.914693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.915627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:21.916562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:21.917797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:21.918808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:21.920274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:21.920274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:21.920274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:21.920538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 252/505] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.921383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.923694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 254/506] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:21.923694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:21.931402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.023618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.027449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35a08-dbf5-419f-b832-1af8605be47a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.029678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63e1e-f2a9-4ab6-8282-3a83bf34a04d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.060731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.061395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.061680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.061956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.062478 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.062677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:22.062984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.063082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.063343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.063614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.064354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:22.064449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.064659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.066102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.102203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 253/507] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:22.102766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 255/508] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:22.103729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.104259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.167110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.204864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f9914-5dcc-4365-b867-749abe7add2c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.206402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b65cd-fb2c-4cfe-bb64-4d6c15569e92 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.242360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.242587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.242970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.243067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.243256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.243346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.243797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.243915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.244299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:22.244709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.246060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.255671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:22.260866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:22.261825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:22.262650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:22.263257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 256/509] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:22.263683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.275568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 254/510] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:22.275568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.299015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.378644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac3f58-174f-49d6-9769-7f89ff26c1d6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.390099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a87805-a084-43da-a4a0-264c9dc4b5af None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.401752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.414104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.425091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.425581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.425581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.425581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.425581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.429105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 257/511] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:22.431118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.494909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.496395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.511529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.534626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4aa3d4-106a-4ec4-b7e3-c714bb19c843 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.569806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.570443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.570727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.571003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.597905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 255/512] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:22.598746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.624681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.643729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.645351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.702346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ca654-4c57-4a36-b766-994e7d597fa9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.716801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.737605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.738215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.738501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.738783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.739450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:22.739827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.741209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.743088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 258/513] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:22.743962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.751071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 256/514] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:22.751810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:22.824032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:22.841238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6786a-234e-4a00-b607-e30d20391b6c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.855083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0beef-c3e0-4777-9fb8-62407fed3e3f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:22.875476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.876220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.876503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.876776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.889577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:22.890182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:22.890468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:22.890747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:22.891394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.892862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.944546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:22.946063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:22.975938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 257/515] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:22 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:22.976840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.016192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.048279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.074509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 259/516] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:23.074509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.110237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407c5-ee58-4b00-a1e1-ad6fd8a101b0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.153008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.153670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.153956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.154243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.170105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.172223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:23.192121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:23.193053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:23.193912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:23.193912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.193912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:23.196236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 258/517] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:23.201917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.202066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8f732-0f4b-465f-a396-a224a4e565c9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.245199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.245857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.246142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.246419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.259951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.261619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:23.281228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:23.282200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:23.283059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:23.283059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.283059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:23.283059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 260/518] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:23.283279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.294341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.313197 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81f18-86f1-412c-adc8-7f9604c1de50 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.321722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.363813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.364464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.364747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.365022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.376003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.377777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:23.395204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:23.396217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:23.397077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:23.397077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.397077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:23.397077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 259/519] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:23.397291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.414770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a34e-e628-4501-b91f-346021e2f36e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.449202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.449757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.450045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.450324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.461320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.462888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:23.477669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:23.478581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:23.479412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:23.479412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:23.479412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:23.479412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 261/520] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:23.479680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.520906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe93ac-e6d3-4e26-a38a-a5a624080a3f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.560021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.560628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.561321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.561599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.562553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:23.562861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.564529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.567237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.567884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.585518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 260/521] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:23.585518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.585783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1417d8-8b90-4b61-9643-857991e18177 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.617824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.618428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.618714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.618991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.619707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:23.620019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.621398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.637880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 262/522] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:23.637880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.700524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167970-c420-4a41-aad1-a78907564db7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.736174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.736946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.737228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.737818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.738468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:23.738776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.740198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2d01-3ac0-4c3c-9bd3-997ada8167d5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.740312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.752474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 261/523] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:23.757614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.779573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.781080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.781363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.781635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.782272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:23.782574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.783962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.797487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 263/524] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:23.798347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.808274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.826056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:23.876566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ea312-7ca7-4cc0-8bcc-7effef804e06 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.918865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.919591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.919911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.920192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.932323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.933292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ce476-80b1-4ee9-afca-69d83daf9f48 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:23.933798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:23.946250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:23.947629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:23.949174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:23.952260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 262/525] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:23.953102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:23.976108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:23.977759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:23.978071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:23.980197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:23.988143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:23.989626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:24.003920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:24.004800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:24.005656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 264/526] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:24.006114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.070718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3f4b-a798-4f05-a276-e6af26c36988 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.081571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.093271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.111693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.117723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.117723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.117723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.121897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.123356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.126433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358d95-b022-4d8a-999c-928d6735f97a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:24.141855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:24.143041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:24.144266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 263/527] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:24.145349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.182171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.182861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.183147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.183412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.191338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.192993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:24.207786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:24.213913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 265/528] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:24.214890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.304310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288350f-1a1f-4e7a-a17e-e29e10333e43 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.335401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbc493-921f-4c5a-bb92-a38f93ed9d3f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.350517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.351136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.351406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.351713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.352401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.352736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.354137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.365292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.373337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 264/529] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.374711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.381518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.382136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.382424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.382700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.383404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.383743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.385239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.405967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 266/530] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.406822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.422718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.500785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6292e-f948-483e-a7ad-fd45ca7b454b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.518170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34010e60-60de-45e3-b1e1-a02929449b10 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.540292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.540916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.541199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.541476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.542206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.542520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.544473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.555192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.555854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.556143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.556427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.557245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.557563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.559020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.564819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 265/531] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.565018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.582249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 267/532] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.582249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.633407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.672781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:24.680168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426057-6cab-4ac8-b423-43837aa7191d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.690141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4604-b9e2-48bb-b58a-7d8039d03bbe None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.717915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.718528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.718814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.719129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.719852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=37e3dbce9a544a519e965dc35cd5ba9f) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.720454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.721628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.735233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 266/533] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/role_assignments?user.id=37e3dbce9a544a519e965dc35cd5ba9f&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.735233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.735511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:24.735511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:24.735511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:24.735511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:24.735874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:24.735874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:24.735874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:24.749293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 268/534] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:24.751027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:24.918963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350dac2d-9f66-4892-b8d9-a6864540b17a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.968464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4784b1-f232-4101-b63e-45c76ba40dae None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:24.987140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:25.014665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:25.014665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.014665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.014665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.042007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:25.043178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.043686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.044166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.045125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=2de7b02bd5864be1b8e8bc6e06dddd55) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:25.046390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:25.048227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:25.054203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:25.063203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 269/535] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/role_assignments?user.id=2de7b02bd5864be1b8e8bc6e06dddd55&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:25.109573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:25.111498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021920bf-27b1-4e31-b2b4-ceff8c2be035 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:25.234015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 267/536] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:25.241976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:25.331715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6513b-78e7-4931-9ecf-4ce3dc65a609 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.332014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6513b-78e7-4931-9ecf-4ce3dc65a609 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.332285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6513b-78e7-4931-9ecf-4ce3dc65a609 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.333927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 268/537] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:25.339776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.340224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.340518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.361927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:25.363331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:25.385260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b833ad-e894-4d51-ba4a-51f63d179e8d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:25.428909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:25.429705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.429994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.430271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.439025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:25.453754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:25.459638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:25.461656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 269/538] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:25.483865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:25.485939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:25.488062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:25.488062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:25.488062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:25.488062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 270/539] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:25.492301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:25.621207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb133fc-095b-48b6-ba2d-f3948f3be528 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:25.657390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:25.658001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.658283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.658559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.659203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:25.659510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:25.660909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:25.673165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 270/540] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:25.678845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:25.710500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:25.794113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bdaf-5cdb-4940-83cb-8ebb1584e831 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:25.830922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:25.831576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:25.831890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:25.832153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:25.839724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:25.841219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:25.854055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:25.856995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:25.860626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 271/541] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:25.864789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:25.889443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:25.945263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:26.001584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d7e3-7b36-41c9-a53f-64c59da3caf3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.056014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:26.060768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.061984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.062281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.062629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.063342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:26.063672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:26.065137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:26.085309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27626e-6073-41c0-9083-8d148702c887 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.098248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 271/542] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:26.104173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:26.159550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.160151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.160438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.160716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.186097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:26.194675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:26.195586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:26.196419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:26.197500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 272/543] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:26.203061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:26.313953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e07d-7986-4652-8f05-afe12930f295 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.334188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:26.361537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419f1c4-aa53-4663-b97d-4e2cc28691f5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.402155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.402775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.403060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.403341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.404271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=ed6b2414b84f4ba0a59b3700b122ba66) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:26.404674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:26.409753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:26.410372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.410993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.411348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.411680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.412501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:26.413017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:26.414311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:26.423163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 272/544] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:26 2022] GET /identity/v3/role_assignments?user.id=ed6b2414b84f4ba0a59b3700b122ba66&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:26.430521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 273/545] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:26.437967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:26.595517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f45a6e-5699-4c3e-8ebc-c9f33d5d08f5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.646057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.646701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.647030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.647314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.664489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:26.665868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:26.681869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:26.682902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:26.684367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:26.684367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:26.684367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:26.684367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 273/546] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:26.689408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:26.701851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:26.802587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2036-8048-4a50-adb7-7a0510f58262 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:26.839071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:26.839764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:26.840056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:26.840406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:26.841230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:26.841595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:26.843723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:26.856842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 274/547] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:26.863018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:26.897973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:27.005744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a447cf-0377-4070-b12c-49c646875c6e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:27.056448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:27.057099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.057385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.057663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.067717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:27.069189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:27.082372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:27.083325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:27.084727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 274/548] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:27.089957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:27.166367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:27.203153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa35b1c-5eba-4cbf-b0bc-d173a141851d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:27.240329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:27.240954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.241237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.241510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.242257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:27.242565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:27.244038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:27.263611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 275/549] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:08:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:27.269941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:27.385461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:27.407611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa312-c76d-4023-8a78-3a86f0117d09 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:27.461898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:27.462511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.462808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.463085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.556486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:27.558015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003fdaf-ea01-44c6-96b7-3895cdd891a1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:27.687789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:27.707367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 275/550] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:27.713301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:27.720515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941f4b1-63bb-463b-88a2-3671d8d6571e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.720911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941f4b1-63bb-463b-88a2-3671d8d6571e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.721067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941f4b1-63bb-463b-88a2-3671d8d6571e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.722967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 276/551] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:27.729082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958711-7e1c-4831-8f68-6aeabd528823 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.729379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958711-7e1c-4831-8f68-6aeabd528823 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.729640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958711-7e1c-4831-8f68-6aeabd528823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.753401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e7958711-7e1c-4831-8f68-6aeabd528823 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:27.754698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e7958711-7e1c-4831-8f68-6aeabd528823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:27.828251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 277/552] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 15 19:08:27.829602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51642-7c7d-47be-947e-c4329e61705b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.829895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51642-7c7d-47be-947e-c4329e61705b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.830170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51642-7c7d-47be-947e-c4329e61705b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.831760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 278/553] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:27.841037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.841341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.841589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.851522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476ba43-40e7-4e91-b3d1-4c698760c4f4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:27.882959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:27.883150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:27.884212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:27.899799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:27.900456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:27.900740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:27.901023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:27.935685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:27.935685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:27.966818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:27.967857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:27.968723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:27.968723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:27.968723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:27.968723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 276/554] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:27.975118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.066493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 279/555] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:28.111979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.145211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5909431-569e-4556-94b8-ccf3fe7782da None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.224231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.224760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.225046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.225329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.225985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:28.226294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:28.228047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.233452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f40f6-81e1-466a-982b-e9f235831db8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:28.254860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 277/556] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:28.256195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee50a8-f616-4421-8bf7-06441469e702 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.256486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee50a8-f616-4421-8bf7-06441469e702 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.256770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee50a8-f616-4421-8bf7-06441469e702 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.257994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 278/557] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:28.266043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee50a8-f616-4421-8bf7-06441469e702 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.307236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5d60d-0cbd-4e5e-be04-ba9efcb289e9 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.310775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.344683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.345770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.346250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.346722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.358129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-1fa325da-c3cd-4f5c-af63-85be3ef0ec4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:28.363623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:28.365019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:28.366073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:28.369694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 280/558] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:28.372160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.372977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.374158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.388953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee50a8-f616-4421-8bf7-06441469e702 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.394928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:28.396140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:28.440361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.441320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.442274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.442809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.452902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:28.455565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:28.476892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:28.477985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:28.479071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:28.481764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 281/559] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:28.482802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.486511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 279/560] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:28.488085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.535491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.606908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.611315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb97ac-77be-452c-bc71-36c9116c3f16 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.616502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2143-8eeb-4ac2-92f9-03454288a978 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.653952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.655440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.656233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.656695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.662112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.662876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.663160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.663437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.664164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:28.664473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:28.666025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:28.669636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:28.674811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:28.675828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:28.676663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:28.679634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 280/561] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:28.679990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 282/562] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:28.680844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.685194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.801007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556c65-1c78-40e3-bd33-b71b77d44daa None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.813923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.821094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3662d-2e37-4a07-8254-d42fc3619186 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:28.837915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.838607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.838892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.839165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.839877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:28.840182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:28.841595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:28.859885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 281/563] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:28.859885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.865418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:28.866073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:28.866355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:28.866628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:28.878427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:28.879909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:28.881232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:28.895919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:28.896943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:28.902836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:28.902836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:28.902836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:28.902836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 283/564] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:28.902836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:28.992316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c0dc-2248-4ba3-8fc3-eb1ab74f6225 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.003192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464598c3-b427-4d45-a0f6-879c4c51eea2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.025377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.026068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.026428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.026772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.027504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.027912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.029284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.036359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.036980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.037257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.037525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.042346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 282/565] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.042346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.048612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:29.053924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:29.054978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:29.056005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 284/566] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:29.056841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.066544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.124237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.142741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27caf-8468-467c-a3d0-e87816b00f1f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.158188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e06d-2c58-481e-9f95-f878cbd9085c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.183273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.183911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.184251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.184580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.185299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=e0c4e352d9914c3a9a5225439274d91d) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.185603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.187054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe98514-712c-46f5-ae9c-0f23cde39549 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.192845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 283/567] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/role_assignments?user.id=e0c4e352d9914c3a9a5225439274d91d&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.194080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83597b-ab64-4194-a085-3875eb3c38b0 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.194419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83597b-ab64-4194-a085-3875eb3c38b0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.194696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83597b-ab64-4194-a085-3875eb3c38b0 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.196082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 284/568] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:29.196223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.196766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.197052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.197333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.198004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.198310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.199775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.201033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83597b-ab64-4194-a085-3875eb3c38b0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.222786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 285/569] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.222786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.312706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.337441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83597b-ab64-4194-a085-3875eb3c38b0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.353339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1cdb-faee-488c-9aee-a63c55448505 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.380694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.381345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.381630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.381910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.393959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.395548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.398598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.402128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.402751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.403546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.403938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.404539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.405967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.407105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c742f-4e51-4409-aa91-2f5b4db07812 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:29.416691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:29.417678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:29.424327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:29.424327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:29.424327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:29.424685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 286/570] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.424685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.424685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.424685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.425171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 285/571] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:29.425171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.446713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:29.447968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:29.526419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 287/572] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:29.527582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.563400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9125ee-8336-4da6-b6c9-d353a9273885 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.593866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.612899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.614299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.615111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.615909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.630092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.632555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.638818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35b6dd-0ce8-4efa-899b-5b95383df08a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:29.649506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:29.650516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:29.651568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:29.655885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 286/573] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:29.656993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.694564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.695354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.695790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.696067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.696766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.697075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.699222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.716166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 288/574] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.717069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.730392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.781142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d911f1-ff7d-43eb-accb-2f5bbe03f4b9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.817722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.818638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.819108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.819567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.830556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.832307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:29.847676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:29.848902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:29.850073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:29.850073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:29.850073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:29.850670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 287/575] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:29.851501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.861032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b6dd-330a-443d-b49b-2dad0d64f046 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.884487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:29.900286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:29.901078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:29.901373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:29.902769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:29.902769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:29.902938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:29.905725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:29.929991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 289/576] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:29.930872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:29.962166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a1acc-4960-4dc7-be8b-eab3f801fdb4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:29.972712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.007352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.008187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.008479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.009258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.018423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.020057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:30.032437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.033875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 288/577] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.035471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.051847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff70-daf5-40d9-9ab0-4c249b548819 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.101765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.102375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.102620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.103333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.115576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:30.121276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.122226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 290/578] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.128588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.158279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda6fe-a7a8-47f7-b684-689bf8288063 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.172520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.193950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.194704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.194968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.195254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.195938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:30.196247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.197652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.211624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 289/579] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:30.212339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.244803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.258716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6125-3c85-4f22-ad4a-b5ed64642894 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.296108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.296733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.297017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.297291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.307889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.309351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.318260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272469d-6c48-41f0-9bc5-883c0ef964e1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:30.324773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.325763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:30.327190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:30.327190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.327190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:30.327190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 291/580] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.328076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.362847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.363381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.363669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.363966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.364702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:30.365007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.366448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.385852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 290/581] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:30.391206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.427529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.437928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982cc5b-0024-4498-a476-8f64be917d6a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.475786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.476392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.476677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.476955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.477625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:30.477935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.479300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.486918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.493600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 292/582] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:30.493600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.509058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad5d63-dd21-4f0f-bf7b-06cacd3a9d4f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.550264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.550264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.550264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.550264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.559458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.561159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:30.576433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.577469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:30.578415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:30.582749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 291/583] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.583514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.604069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8a50-a4fc-4fa9-9b37-61603b2d49f3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.652206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.652757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.653040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.653312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.654084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:30.654477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.656863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.674948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 293/584] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:30.674948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.691307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.700340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3038d-bc01-48cd-809f-61d22b5cedf9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.749042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237489-5cdd-4681-867a-381552736184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.749989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237489-5cdd-4681-867a-381552736184 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.749989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237489-5cdd-4681-867a-381552736184 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.750285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237489-5cdd-4681-867a-381552736184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.766939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1237489-5cdd-4681-867a-381552736184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.768481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1237489-5cdd-4681-867a-381552736184 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.775486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a1237489-5cdd-4681-867a-381552736184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:30.785943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.792831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:30.793743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:30.793743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.793743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:30.793743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 292/585] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.793743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237489-5cdd-4681-867a-381552736184 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.808475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167865-2bb5-4539-bf48-4cdfac7a8656 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.854477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.855215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.855815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.856170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.870684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.872253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:30.887386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:30.888676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:30.890086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:30.890086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:30.890086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:30.893893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 294/586] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:30.894469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.909651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237489-5cdd-4681-867a-381552736184 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:30.948327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:30.948945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:30.949227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:30.949493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:30.950353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:30.950820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:30.953070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:30.975208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 293/587] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:30.975785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:30.985454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.035089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399326e-b9c8-45ae-a674-9f870b12dd78 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.045110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.090304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.090934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.091632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.091632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.109854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.111546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.118469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8e4c-badb-41da-9e13-936a6065be80 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:31.131527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:31.132727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 295/588] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:31.133612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.158946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.159607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.159973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.160244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.160882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:31.161209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.162569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.174291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 294/589] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:31.179468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.245761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68e37-a9b2-4713-9f47-29cdc91e969c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.275723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.287530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.288271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.288560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.288830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.289669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:31.289792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.291150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.300849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91300-5172-42d8-a185-ca96029fd978 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.309161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 296/590] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:31.309161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.325530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.346335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.347024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.347412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.347941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.360158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.361640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:31.377307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:31.378364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:31.379276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:31.379276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:31.379276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:31.380846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 295/591] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:31.381674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.416441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35842-b5f9-4f19-b8ff-449400bf314b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.450540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.451144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.451575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.452534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.453377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:31.453831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.455397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.467076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 297/592] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:31.467944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.482575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28419-8691-4c70-bc02-65f4e0801f0a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.515207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.520222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.520944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.521302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.521648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.531251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.534061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9738266f-1445-444d-8ba5-33ac7a707c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:31.554437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:31.555595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:31.556606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 296/593] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:31.558180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d01dc-75d8-43a3-a26f-47b1ae773b65 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.558625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d01dc-75d8-43a3-a26f-47b1ae773b65 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.559013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d01dc-75d8-43a3-a26f-47b1ae773b65 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.560428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 297/594] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:31.562385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d01dc-75d8-43a3-a26f-47b1ae773b65 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.596301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f1e1-b87d-485f-bb95-12bb335f2862 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.601095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.632010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.632632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.632913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.633182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.640649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.642079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:31.653291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:31.654489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 298/595] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:31.655986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.682430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d01dc-75d8-43a3-a26f-47b1ae773b65 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.717412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.718019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.718296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.718568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.719204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:31.719509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.720922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.733711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 298/596] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:31.734543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.763245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8b5f-ae96-4e61-a670-486588f71793 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.766488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.799970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.800637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.800922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.801203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.816068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=c9b27eb952a04c46bb36653fb79abe3e, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.817555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.822719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:31.834230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4e267-3f9d-4e1c-ac91-75a3461030c7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:31.867916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:31.868515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.868791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.869059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.869737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:31.870037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:31.871485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a140d-c9c8-4a73-914c-07fba6db57fd None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:31.883818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 299/597] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:31.885054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:31.885337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:31.885969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:31.890151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 299/598] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:31 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:31.891066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:31.912323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:31.913579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:32.000305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 300/599] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:32.001020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.047305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fe6cd-e778-42bf-a1d9-076ace2ca1c6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.091675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.093256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.093548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.094129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.094983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:32.095162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.096632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.114691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.125508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 300/600] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 15 19:08:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:32.127353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.141023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ced2ed-ce71-4978-b26d-a28682a231a4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.151015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.181163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.181962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.182348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.182740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.190622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.192352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:32.204882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:32.205961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:32.206881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:32.207309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 301/601] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:32.208309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.259468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f3834-0f70-4f26-afb5-353db00df31d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.298019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.298687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.298970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.299244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/37e3dbce9a544a519e965dc35cd5ba9f/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.315687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=37e3dbce9a544a519e965dc35cd5ba9f, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.317470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.324590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811aed0-1948-4ab0-b0a4-bd5243f87148 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.361568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.362192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.362471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.362745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.370600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.374488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.375984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:32.391240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:32.392439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:32.393795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:32.393795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:32.393795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:32.393795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 302/602] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:32.393795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.449639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 301/603] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:32 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/37e3dbce9a544a519e965dc35cd5ba9f/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:32.450604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.523666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0992b1-33bf-4bc8-a2a1-0bb2cafed626 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.532511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.577137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.578108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.578229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5269a5-3d4d-4d4b-94a3-5bc94df2545b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.578844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.579306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.581022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:32.582350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.584084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.604349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 303/604] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:32.606301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.631206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.631899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.632184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.632461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.645354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:32.650780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:32.652056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:32.653263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:32.658963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 302/605] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:32.658963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.697233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.770392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185037b-f492-4cae-a040-215d938890e4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.800216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20c20-6545-425d-869a-5c3a96df01e6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.818018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.818917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.819205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.819486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.820480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:32.821187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.822582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.828018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:32.841683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 304/606] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:32.842732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:32.847215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:32.848057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:32.848408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:32.848752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/2de7b02bd5864be1b8e8bc6e06dddd55/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:32.867615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=2de7b02bd5864be1b8e8bc6e06dddd55, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:32.869724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:32.970406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268055e7-d19a-4967-a3f2-3aab3f8adb45 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:32.975080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 303/607] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:32 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/2de7b02bd5864be1b8e8bc6e06dddd55/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:32.978463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.002073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:33.076560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.077197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.077390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.077672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.078361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:33.078670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.080118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.099539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 305/608] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:33.105145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.119113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676b5d5-8ad6-4428-9a9d-6074fed92614 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.164810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.166975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.167770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.168282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.178888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.180664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:33.197430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:33.198472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:33.199518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 304/609] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:33.202587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.245266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:33.317100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdd38-0e46-4114-82a5-fb993a8f2295 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.332683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8d517-147b-49d8-9944-919e44b9da73 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.382856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.383492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.383806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.384082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.391541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.392175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.392460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.392735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.393497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:33.393807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.395223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.407349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.413279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.420982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 305/610] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:33.427477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:33.442651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:33.443783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:33.444684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:33.444684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:33.444684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:33.445497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 306/611] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:33.452271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.503333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:33.551149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:33.560580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fbb9f-fcd6-4154-a257-76a7b9e9f195 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.597339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.598020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.598308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.598589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/ed6b2414b84f4ba0a59b3700b122ba66/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.602907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215a4b3-bd85-4f2c-bf4e-43b3cf37e7c0 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.614914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=ed6b2414b84f4ba0a59b3700b122ba66, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.616409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.657157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442713a-7080-4160-927d-494bf496973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.657815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442713a-7080-4160-927d-494bf496973a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.658147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442713a-7080-4160-927d-494bf496973a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.658424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442713a-7080-4160-927d-494bf496973a None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.659065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442713a-7080-4160-927d-494bf496973a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:33.659371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442713a-7080-4160-927d-494bf496973a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.660887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442713a-7080-4160-927d-494bf496973a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.674455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 307/612] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:33.684184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442713a-7080-4160-927d-494bf496973a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:33.689174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 306/613] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:33 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/ed6b2414b84f4ba0a59b3700b122ba66/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:33.814764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:33.912943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442713a-7080-4160-927d-494bf496973a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:33.964029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:33.965071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:33.965571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:33.966058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:33.974515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:33.976282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:33.989546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:33.990569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:33.991540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:33.994139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 308/614] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:34.000182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:34.146567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857feaf6-1422-42df-9d93-971abeecff6f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:34.202923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:34.203548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.203812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.204037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.204788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:34.205097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:34.206579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74bd4b-5124-424f-8136-9cf1972c1278 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:34.229083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:34.235830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 307/615] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:08:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:34.239546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:34.411685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33567-7150-4f41-acd5-0a912b328f67 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:34.447661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:34.448307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.448591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.448868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.460768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:34.462284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:34.465069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:34.474068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63544-1617-428d-97ec-cf9174a3c140 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.474505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63544-1617-428d-97ec-cf9174a3c140 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.485774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63544-1617-428d-97ec-cf9174a3c140 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.485774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 308/616] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:34.485990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:34.486842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:34.487936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:34.487936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:34.487936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:34.487936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 309/617] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:34.488179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.488179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.488179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.488379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:34.513293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:34.515044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:34.649016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d167-b4ba-4ef4-bcb9-15ea8762d475 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:34.658314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 309/618] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:34.699084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:34.699732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.700020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.700290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.700938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:34.701245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:34.702891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:34.720542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 310/619] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:34.724910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:34.859945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ef2b6-5dec-438a-95e5-1805e074de1e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:34.905214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:34.915988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:34.916635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:34.916919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:34.917198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:34.934637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:34.936651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-763290c0-381b-4d54-a6ad-4ba875b2baa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:34.954159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:34.955099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:34.956033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:34.958580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 310/620] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:34.964250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.052656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:35.112000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be86d-2d39-4d19-aa69-3fd28911717f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.131296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eae16-46d6-42ca-b359-81e5de1c4bcf None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.131695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eae16-46d6-42ca-b359-81e5de1c4bcf None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.131879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eae16-46d6-42ca-b359-81e5de1c4bcf None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.133215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 311/621] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:35.140357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.140357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.140357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.166584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:35.167570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.168099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.168571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.169480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:35.169986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:35.171704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:35.172696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:35.173911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:35.200318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 311/622] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:35.206482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.303949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 312/623] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 19:08:35.304738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.350683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85f4d-da2f-4e33-b777-398691e4f8e6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.369537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:35.390541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:35.391316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.392588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.392879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/e0c4e352d9914c3a9a5225439274d91d/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.413667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=e0c4e352d9914c3a9a5225439274d91d, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:35.415637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:35.482821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87ef9b-1e76-48cf-b200-9b7db70a3651 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.502094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 312/624] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:35 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/e0c4e352d9914c3a9a5225439274d91d/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:08:35.506190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:35.522526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:35.523279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.523569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.523888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.537206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:35.542296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:35.546442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:35.547316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 313/625] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:35.550668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.617312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.691632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4eb6-7d0f-440d-a823-d576f4eb897e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.733609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df753e9-3b0b-4873-bc71-8655c92e2ff3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.737023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:35.737642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.737919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.738191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.738859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:35.739162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:35.740554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77912b-3d45-4cf0-9dc5-16bb1336fc00 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:35.751812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 313/626] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:35.752560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4649f-ed9d-47d8-87b8-c30f0d84b974 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.754467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4649f-ed9d-47d8-87b8-c30f0d84b974 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.754467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4649f-ed9d-47d8-87b8-c30f0d84b974 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.754757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 314/627] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:35.759561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4649f-ed9d-47d8-87b8-c30f0d84b974 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:35.777461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:35.778090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.778371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.778644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.790346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-6a83e5a0-6b51-478d-83f2-8d07e6da506f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:35.797023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:35.798013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:35.798867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 314/628] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:35.799803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:35.800553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:35.800553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:35.820824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:35.822004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:35.822345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:35.866013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:35.913886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4649f-ed9d-47d8-87b8-c30f0d84b974 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:35.920830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 315/629] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:35.922394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.008394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.009036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.009321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.009601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.033827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.035470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.052765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26880-f16c-482c-98f1-9d71d5ed2495 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:36.064461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:36.065552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:36.066452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:36.066452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:36.066452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:36.067513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 315/630] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:36.079010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.097249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.098263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.098695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.099122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.099988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:36.100433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.102600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.113609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 316/631] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:36.120887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.180320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.244083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.266460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca130-bb6a-43e7-b532-374f3ad665c7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.270777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f573933-b8ab-4a89-9d03-eaabf173094b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.302471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6508f6-3468-4449-b31d-61246155664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.303102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6508f6-3468-4449-b31d-61246155664d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.303383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6508f6-3468-4449-b31d-61246155664d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.303681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6508f6-3468-4449-b31d-61246155664d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.308040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.308650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.308935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.309212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.309859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:36.310227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.312809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.316133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6508f6-3468-4449-b31d-61246155664d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.317604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6508f6-3468-4449-b31d-61246155664d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.326921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 316/632] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:36.328318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-df6508f6-3468-4449-b31d-61246155664d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:36.334044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:36.334971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:36.335893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:36.335893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:36.335893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:36.335893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 317/633] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:36.336658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6508f6-3468-4449-b31d-61246155664d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.427112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.443354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6508f6-3468-4449-b31d-61246155664d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.443781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ec3c-19db-4a41-92a4-a8c999f4db19 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.478857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.479512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.479833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.480108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.480482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.481323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.481810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.482267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.489953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.492160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.493621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.493738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:36.498794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:36.500040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:36.500891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:36.504873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 317/634] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:36.505735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-d67b8227-c2d6-48ab-9541-31014ed40112 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:36.507677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:36.508619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:36.509558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:36.511953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 318/635] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:36.512990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d2e5-5a9d-43b4-b7bb-ff004a869c3f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.513272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d2e5-5a9d-43b4-b7bb-ff004a869c3f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.513546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d2e5-5a9d-43b4-b7bb-ff004a869c3f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.514825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 319/636] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:36.515504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d2e5-5a9d-43b4-b7bb-ff004a869c3f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.617998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd29c06-3a66-44ff-8d81-76b50cb93dcc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.628592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d2e5-5a9d-43b4-b7bb-ff004a869c3f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.650507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.651110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.651390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.651681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.652316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:36.652618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.653991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.665678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 318/637] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:36.666453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.673589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.674385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.674671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.674949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.675612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:36.675963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.677315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfb9ac-7755-4ffb-ba3b-116dc435f863 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.690816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 320/638] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:36.690816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.690816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.690816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.702325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.708497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:36.710444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:36.734897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.776001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb82f3-4812-43ac-af26-47c978853034 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.795678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 321/639] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:36.795678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.828212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.828824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.829104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.829372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.830099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:36.830496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.831909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:36.849960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 319/640] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:36.850794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:36.937257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944b14b-da65-486a-88d4-ceeaf7666e02 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.974507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:36.975451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:36.976082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:36.976533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:36.976637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99673-04f9-455f-9165-a3f697887cd0 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:36.980372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:36.986427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:36.988211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.006915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:37.007039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:37.007950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 19:08:37.008875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:37.010204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 322/641] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:37.011212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.016171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.016800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.017079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.017349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.028961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.030808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:37.047157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:37.048219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:37.049123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:37.049123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:37.049123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:37.053552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 320/642] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:37.053989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.121977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288affd-5f59-4aa9-bcdf-4371d9db2575 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.159924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.160532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.160814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.161088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.161986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56b3bd-bb66-4193-afbc-7fb78dafdd63 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.196273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867957-9daa-497d-9f56-de99459a45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.197027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867957-9daa-497d-9f56-de99459a45af None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.197319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867957-9daa-497d-9f56-de99459a45af None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.197595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867957-9daa-497d-9f56-de99459a45af None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.199095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867957-9daa-497d-9f56-de99459a45af None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:37.199464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867957-9daa-497d-9f56-de99459a45af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.200974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867957-9daa-497d-9f56-de99459a45af None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.224298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 321/643] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:37.224298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867957-9daa-497d-9f56-de99459a45af None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.232219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.240766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.242397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.251285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.329818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867957-9daa-497d-9f56-de99459a45af None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.355791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 323/644] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:37.356651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.376778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.377393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.377670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.377941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.378579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:37.379163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.380280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.395778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 322/645] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:37.396931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.454591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.460660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def884a-faa2-43a3-a448-5bf44e579e06 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.497354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.499034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.499608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.500093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.500379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.509175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abebb-cba6-4386-bed2-458301a5eb6f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.513265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:37.518913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:37.520139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:37.521147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 324/646] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:37.521879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.551973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.552585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.552872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.553141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.624997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.626453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.632622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56add422-9141-436a-8e10-7e5f5a53de0a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.669502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.670717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.670997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.671279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.682895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.684411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:37.701986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:37.703004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:37.703961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:37.703961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:37.703961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:37.703961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 325/647] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:37.703961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.720030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.724251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.737195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 323/648] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:37.742253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.817493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515b4a-5d3c-4126-bb96-b2b8d3877a3b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.842540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62a4c-5511-4a64-9f1c-ea4953ff8d49 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.852294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.852914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.853193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.853498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.861894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.863336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:37.874846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:37.875832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:37.876793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 326/649] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:37.877399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.882300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:37.882939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:37.883229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:37.883506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:37.884210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:37.884521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:37.885898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:37.894376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 324/650] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:37.895183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:37.946466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.949110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:37.981488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad6e-2a65-4b7a-b837-9bb24fcf6e92 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:37.995225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f2e-aab2-4d83-8d44-ea215f1155db None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.014085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.014686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.014966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.015256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.015930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:38.016236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.017591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ee228-bab8-4713-9ce3-b3278dcd1129 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.027939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.028569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.028849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.029280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 327/651] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:38.029890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.031056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008ae87-3189-403c-98a1-917c08797edd None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.031337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008ae87-3189-403c-98a1-917c08797edd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.031610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008ae87-3189-403c-98a1-917c08797edd None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.032830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 328/652] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:38.037634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008ae87-3189-403c-98a1-917c08797edd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.043387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.044878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:38.059891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:38.061104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:38.062178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:38.062178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:38.062178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:38.062178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 325/653] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:38.062453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.146849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008ae87-3189-403c-98a1-917c08797edd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.166510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e936-a4a7-44a7-8327-f542b2246d0b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.168544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.175836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.183270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.183904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.184184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.184457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.185179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:38.185481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.186941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.201260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.201910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.202222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.202880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.208030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 329/654] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:38.208847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.214267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.215747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-be86f48c-0d77-416e-8772-1b85ac7043e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:38.230241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:38.231264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:38.232215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:38.232215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:38.232215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:38.232215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 326/655] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:38.233106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.233368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.233668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.251002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:38.252236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:38.312512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534377-d271-4f8f-a976-0e75ba88beed None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.317343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 327/656] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:38.318179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.359669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.360294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.360573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.360847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.367554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.369008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:38.380110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:38.381180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 330/657] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:38.382072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.412825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.428183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cedd4-e7eb-4068-abb6-5181e8450e4f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.432608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.463476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.464119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.464826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.465111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.465819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:38.466489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.468762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.484240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 328/658] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:38.484240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.494726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1049496-c507-4a03-916e-edc046f0cfe9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.531465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.532209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.532488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.532877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.585785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaaac2-7719-49a6-85b6-108fe7b7daba None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.605631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.607118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.619215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.619860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.620145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.620421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.621056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:38.621361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.622726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.634873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 329/659] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:38.635786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.639791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.657446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.711332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 331/660] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:38.712124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.738832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d492063-8dee-46eb-b629-dddec1aff0a1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.773327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.773930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.774216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.774487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.775174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:38.775488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.776945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.788404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 330/661] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:38.789209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.809343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074d3a8-767c-4f3e-a3ac-7fc4b9086394 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.843326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.843973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.844254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.844525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.850840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:38.851378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.852308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:38.863961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:38.865066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 332/662] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:38.865967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.874046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:38.892875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57e8e1-fb43-487b-9d63-af9e5591b1f4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.929951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.930582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.930860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.931131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:38.942156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:38.947348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:38.948381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 331/663] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:38.949297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:38.966384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753846ee-dd45-479e-a80c-5e39623f96f2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:38.998778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:38.999391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:38.999689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:38.999961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.007143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.008584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:39.020069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:39.025786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 333/664] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:39.026990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.053055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ebf13-d87f-4499-9e0e-ea0ec3a12651 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.076391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.085857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.087162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.087817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.088100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.088377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.098323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.099796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:39.115326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:39.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:39.117436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:39.117436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:39.117436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:39.117436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 332/665] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:39.117436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.121732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9967f-08bb-4af3-b9cb-bfe23f7b2b2e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.156019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.156631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.157649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.157930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.158734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.159041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.160483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.166788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 334/666] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.167565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.238727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd358a3-b807-4cb2-a199-7422392dd984 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.277587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.278204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.278483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.278754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.279492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.279830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.281528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.287094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9472b4-73fd-4334-af81-0c086a0adbb3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.294818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 333/667] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.296291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.324841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.332309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.332928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.333375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.333668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.334249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.335236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.335548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.336973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.346228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 335/668] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.352663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.406955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d74c9-91f1-443c-8e59-c7e3a4b6eba9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.445093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.445704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.445989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.446266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.446993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.447303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.448786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.467526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 334/669] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.468352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.480367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1098-bbbe-4fc1-b9f1-5e5fd1df881c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.518595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.519193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.519419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.519715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.520430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.520732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.522104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.534909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 336/670] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.535839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.572501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.578661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a753303-1ee0-463f-82ca-2d4410d232ee None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.591306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.614045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.614691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.614963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.615297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.616094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=37e3dbce9a544a519e965dc35cd5ba9f) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:39.616402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.618097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.631993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 335/671] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/role_assignments?user.id=37e3dbce9a544a519e965dc35cd5ba9f&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:39.631993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.658577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983269a1-c455-46cc-ad53-5665dd398c69 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.699604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.700301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.700626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.701074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.716213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.718661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:39.739499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:39.740584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:39.741563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:39.741563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:39.741563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:39.741563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 337/672] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:39.741789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.750307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dd44e-1118-47a9-b30c-87dcc86293a4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.791271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.791839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.792120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.792391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.823636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.866925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbeddf6-1a6a-4c4b-81a4-68f3bdb244dd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:39.869614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.870574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:39.871765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.915275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:39.916566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:39.916566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:39.916566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:39.931239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:39.933543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:39.945417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:39.946344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:39.947229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:39.948354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 338/673] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:39.949324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:39.997582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 336/674] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:39.999077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.074848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3462-d135-4eb5-bb76-5c864b581e8f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.084687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.113363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.114007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.114295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.114573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.115302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:40.115616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.117270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.132688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 339/675] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:40.133718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.142614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee592f7-1091-4140-89fa-8b251f31f5ce None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.161860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.191293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.192302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.192784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.193787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.194949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:40.195456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.197797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.214527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 337/676] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:40.215699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.245469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ba1a-a4dc-4316-9aa0-08dc064823c7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.294493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.296076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.296830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.297315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.311187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.313118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.331507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49dea5-11f9-4913-8f53-c73e3e68efef None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:40.333218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:40.334180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:40.335064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:40.335064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:40.335064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:40.341547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 340/677] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:40.342466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.356915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.381940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.383076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.383558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.384088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.395568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.397803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:40.415130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:40.416135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:40.417076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:40.417622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 338/678] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:40.418359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.429373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.466020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5a34-d4fd-4f9a-bdbd-cba80bbd2a24 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.515159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.516356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.516864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.517148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.517886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=2de7b02bd5864be1b8e8bc6e06dddd55) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:40.518198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.519640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.527772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 341/679] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/role_assignments?user.id=2de7b02bd5864be1b8e8bc6e06dddd55&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:40.528605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.547880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f464f7e-af7e-4450-8a73-8b8c05bd8a5e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.599846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.601120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.601410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.601691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.602346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:40.602654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.605566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.633572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 339/680] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:40.640214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.725029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.728288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46389c-eb2c-4183-83d3-9cd2fdd3febd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.782142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.782761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.783050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.783382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.784157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:40.784511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.785953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.796457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:40.806256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 342/681] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:40.811902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.824532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a45da-7240-4099-839a-2b7c6b453a2a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.874272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.874889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.875172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.876014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:40.886496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:40.888540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:40.905387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:40.906571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:40.907469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:40.907944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 340/682] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:40.915506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:40.941210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd3f09-e3d4-4f12-af03-6968297e7481 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:40.980700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:40.981371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:40.981726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:40.982008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.051012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:41.055891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec68979-95d4-4102-965d-3d63650d9cf3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:41.061764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:41.065308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.066509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.106562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:41.108824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.109106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.109381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.110079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:41.110417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.115781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.141711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 341/683] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:41.147849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:41.254986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 343/684] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:41.260074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:41.340589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e90e5b-39f9-4918-a6fc-a02f2c665400 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:41.412887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:41.413418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.413696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.413969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.414689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=ed6b2414b84f4ba0a59b3700b122ba66) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:41.414996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.422577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.434048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 342/685] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:41 2022] GET /identity/v3/role_assignments?user.id=ed6b2414b84f4ba0a59b3700b122ba66&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:41.441912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422d7eb-7015-49ab-a549-41b426fe8320 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:41.482556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:41.483256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.483543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.483856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.492103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:41.496457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.497929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-7e71377f-be0b-4e5d-acb7-0e2c0a116032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:41.515334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:41.516812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:41.518157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:41.518157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:41.518157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:41.518157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 344/686] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:41.528942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:41.563441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:41.608607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5523-4cda-427b-ba65-e13901cbd475 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.608935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5523-4cda-427b-ba65-e13901cbd475 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.609179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5523-4cda-427b-ba65-e13901cbd475 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.611486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 345/687] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:41.621248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.621796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.622221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.659003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:41.660537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:41.705298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046502-7163-49a7-99b6-9fa20b7a1f24 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:41.763324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:41.764031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.764312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.764584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.765225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:41.765528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.767459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.780304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 346/688] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:41.787782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 343/689] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:41.796725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:41.888679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:41.914703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c463a1b-0453-4508-82fa-76984f345f8b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:41.954511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:41.955141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:41.955479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:41.955739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:41.962548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:41.964020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:41.975874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:41.976823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:41.977732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 347/690] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:41.988768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.096013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.140530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:42.159312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3b8a-abb3-4847-931e-bd098be8322e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.202510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.203410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.203717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.203997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.204980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:42.205826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.207471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352e4bb-1ac7-412b-9531-678b93edfaeb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:42.224072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 344/691] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:42.225768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c7a15-8db5-4966-9574-37ba3e8e28c1 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.226056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c7a15-8db5-4966-9574-37ba3e8e28c1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.226335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c7a15-8db5-4966-9574-37ba3e8e28c1 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.226531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daeab3-aafe-42ea-86d9-b9c07429eafd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.227555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 345/692] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:42.230574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c7a15-8db5-4966-9574-37ba3e8e28c1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.265283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.265910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.266241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.266521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.275025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:42.279666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-ca2f91b2-dc9a-44d1-bba9-0fa2be1772d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:42.285430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:42.293422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 348/693] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:42.294463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.296318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.296318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.316702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:42.318009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:42.370092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c7a15-8db5-4966-9574-37ba3e8e28c1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.407674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 349/694] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:42.407674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.413285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.414179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.414639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.415079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.416037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=e0c4e352d9914c3a9a5225439274d91d) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:42.416518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.418241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:42.432702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 346/695] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/role_assignments?user.id=e0c4e352d9914c3a9a5225439274d91d&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:42.515017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211699-57d1-415b-a311-2bb8d555348c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.548821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:42.555430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.556086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.556368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.556690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.557463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:42.557820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.559248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:42.570206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 350/696] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:42.579981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.593133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:42.704945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbdfb2-48e7-41ed-96bf-b9a1ecf63f74 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.767537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.768426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.768426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.768733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.783741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.785213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:42.791154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:42.805104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:42.807357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:42.809560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:42.809560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:42.809560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:42.809560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 347/697] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:42.812533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:42.893752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:42.933476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ecb4a-ec1d-4668-85a1-7fc086b5d631 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.948567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb924a-977b-4025-97ec-5310e8886011 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:42.973681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.974477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.974808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.975081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.985878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:42.986510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:42.986868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:42.987145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:42.987634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.987844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:42.988175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:42.989104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:42.989539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:43.006016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:43.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:43.009761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:43.009761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:43.009761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:43.009761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 351/699] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:43.010041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 348/699] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:43.013115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.014263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.065272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:43.126910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7ca-d3c9-4ac9-9304-2f8e595a7ef8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:43.131995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e82f43-9529-401c-aa0c-7d13e5249f89 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:43.147483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:43.173860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:43.174497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.174836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.175119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.175819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:43.176126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:43.178034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.180728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:43.181289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.181776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.181776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.192662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:43.194076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 349/700] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:43.194275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.201905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:43.211786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:43.214075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:43.216062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 352/701] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:43.225855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.334171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13463dd6-bac0-4f36-af85-ab06be9f8ba8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:43.354044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57876c7c-a1c5-4fc3-8639-21aeb9eb6a69 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:43.371501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:43.372204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.372490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.372766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.373446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:43.374875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4b96-ac38-4dc3-b6cd-309e26fa2c9b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.376014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:43.394626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:43.395358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.395764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.396134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.397047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:43.397450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:43.398908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef178537-ae05-4fda-a63a-86dfdb62cdee None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.419827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 353/702] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:43.419827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c5eb5-b638-4263-9fed-1a74032b4c27 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.419827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c5eb5-b638-4263-9fed-1a74032b4c27 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.419827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c5eb5-b638-4263-9fed-1a74032b4c27 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.419827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 354/703] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:43.423433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c5eb5-b638-4263-9fed-1a74032b4c27 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.491517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 350/704] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:08:43 2022] POST /identity/v3/users => generated 254 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:43.493027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.493316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.493592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.537972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:43.540657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:43.579562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c5eb5-b638-4263-9fed-1a74032b4c27 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:43.610407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:43.640946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:43.641569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:43.644062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:43.644374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:43.685163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 351/705] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:43.717292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:43.756710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:43.758225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:43.865191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 355/706] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:43.870844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.982439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:43.988460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6fe3a-c34d-431c-935d-e74a9bd9fe65 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.055796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.056574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.056866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.057144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.064925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.080670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.082166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.110054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ec8fd-1f93-4751-952f-983a0624f158 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:44.121038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:44.122057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:44.123045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:44.123045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:44.123045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:44.123045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 352/707] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:44.132706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.146040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.146724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.147013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.147286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.147940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.149330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:44.162275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:44.163365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:44.164266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 356/708] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 15 19:08:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:44.170645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.181645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.283444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b63eb-6b3f-4bb2-8084-d662222b1bf7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.308678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b432-a051-4421-b738-c2996b74c327 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.322299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.322940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.323223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.323496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.324224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:44.324532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.326134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7551eb5-e08e-41c2-bb32-9753f81c0597 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.342025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 353/709] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:44.343519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b463a-2dd2-4a9d-b472-47dd1d01a322 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.343841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b463a-2dd2-4a9d-b472-47dd1d01a322 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.344115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b463a-2dd2-4a9d-b472-47dd1d01a322 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.349825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 354/710] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:44.349825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b463a-2dd2-4a9d-b472-47dd1d01a322 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.448882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.479457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b463a-2dd2-4a9d-b472-47dd1d01a322 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.481708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.494978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.495601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.495907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.496187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.496827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:44.497133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.499118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd13196-62ac-49ce-9df4-37dd4fae2453 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.508896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 357/711] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:44.508896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.508896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.508896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.527470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:44.528751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:44.535038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.536015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.536496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.536985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.544476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.546920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:44.559577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:44.561152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:44.562032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:44.564624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 355/712] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:44.566324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.611959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 358/713] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:44.612936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.753868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b7032-0e5d-46ab-8003-245172aae553 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.764952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9954-16d0-4c77-ac2c-bf861cb6d834 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:44.789942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.815688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.816337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.816622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.816902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.817645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:44.818016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.819538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.844937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 359/714] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:44.846618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:44.847250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:44.847540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:44.847886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:44.848887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:44.849182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:44.850651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:44.853148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.855941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:44.862878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 356/715] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:44.962704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:44.995726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca73be-4ae1-403f-b3f7-20178614fac0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:45.036249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:45.036796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.037085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.037365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.038116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:45.038432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:45.039956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:45.046882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 360/716] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:45.186694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c90f46-835e-4b67-adc9-f13ea5bdd64d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:45.237950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:45.238608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.238897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.239171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] PATH_INFO: `/v3/services/network` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.239879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:45.241281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:45.255876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:45.256985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:45.257918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:45.259206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 357/717] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:44 2022] GET /identity/v3/services/network => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:45.263691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:45.331517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:45.391329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2775-1983-4795-8b87-ef714f125494 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:45.431225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:45.432041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.432425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.432806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.433551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:45.433961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:45.435580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063d803-57f8-4ccc-a3a8-bf5ad9e2def1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:45.441193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 361/718] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:45.452157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:45.540918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30fa6-7bea-437c-b93c-de318787a0c3 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.541782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30fa6-7bea-437c-b93c-de318787a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.541782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30fa6-7bea-437c-b93c-de318787a0c3 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.543426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 362/719] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:45.550018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.550445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.550827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.561038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:45.574980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:45.576651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:45.654038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1557825-64f9-44b9-8d2e-305349d4a93b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:45.661279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 363/720] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:45.710263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:45.711244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:45.714705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:45.714705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:45.714705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] RBAC: Adding query filter params (type=network) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:45.714705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:45.714705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:45.721389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 358/721] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:46.026909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:46.077947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:46.160276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059c8b1-3089-4c9d-866c-12efcd2e6f56 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:46.207667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:46.208231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.208513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.208785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.229706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:46.235265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:46.236280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:46.237126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:46.239267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 364/722] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:46.242559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:46.398404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843bc3-2af5-4d67-b3ab-31edf145f02c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:46.438951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:46.447591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:46.448231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.448514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.448786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.449456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:46.449756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:46.451454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d1910-f85f-41df-a422-fe6b898163a1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:46.462642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 359/723] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:46.472483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:46.518730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b0e5-e3f7-49cb-91f0-a7bccbeca794 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.519035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b0e5-e3f7-49cb-91f0-a7bccbeca794 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.519453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b0e5-e3f7-49cb-91f0-a7bccbeca794 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.520892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 360/724] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:46.528031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.528400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.528745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.564740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:46.567352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:46.652022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:46.654959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5e792-8b31-48b2-938f-de883a768c26 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:46.702425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 361/725] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:46.735803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:46.735803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.735803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.735803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.736529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:46.736529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:46.753563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:46.754778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:46.755733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:46.755733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:46.755733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:46.755733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 365/726] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:46.760702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:46.889353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019515b-ef76-44d3-8491-83249052b2c9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:46.943533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:46.949124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:46.949753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:46.950035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:46.950308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:46.951086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:46.951472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:46.952923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:46.973558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 362/727] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:46.978725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.071617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.143676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aaac3-18c6-4293-9650-bb798a381b59 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.193800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.194450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.194731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.195001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.214064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.215679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:47.240128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:47.241164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:47.242103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 366/728] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:47.245378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c131f2-e96e-4ea5-9bb5-f0b2eaf909d1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.247518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.276067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:47.288415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.289069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.289348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.289621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.302788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:47.309193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:47.312124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:47.313178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 363/729] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:47.320762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.364101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:47.463848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f1de3-90be-452d-b13a-d27e24250d68 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.470541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf322d-91a4-4cb5-bedd-ff80372ce29c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.532372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.532984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.533263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.533541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.534264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:47.534571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.536021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.541589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.542208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.542490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.542811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.543481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:47.543810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.545722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.563639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 364/730] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:47.570204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.580236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 367/731] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:47.586911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.678261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:47.687395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f781-b28f-4ba0-b348-4dbc44512fbf None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.692049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:47.721273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006a2d1-cfbf-4a00-8fab-8bc0c2db6292 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.762683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.763352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.764623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.764623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.776495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.777168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.777451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.777728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.791180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.792729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2df4d9d6-ed76-4f5a-bb35-a2bbc705044c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:47.809001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:47.810003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:47.810925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:47.810925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:47.810925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:47.810925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 365/732] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:47.816046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17a1-de77-46b5-9524-d34f1f117db9 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.816046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17a1-de77-46b5-9524-d34f1f117db9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.816046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17a1-de77-46b5-9524-d34f1f117db9 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.816046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 366/733] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:47.817429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba17a1-de77-46b5-9524-d34f1f117db9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:47.896262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.897741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafc170-2fa6-4f7c-bdeb-4f6c8cca86d0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.938849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba17a1-de77-46b5-9524-d34f1f117db9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:47.972463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:47.977683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:47.978399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:47.978684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:47.978949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:47.979606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:47.979935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:47.981366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bd7d3-6558-4dd4-9094-b95769275f0d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:47.994878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 367/734] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:48.000416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.000416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.000416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.016264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:48.017477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:48.020067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 368/735] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:48.021728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2cbeb-b8ab-4011-b2e1-e307b6f2fbd8 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.022313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2cbeb-b8ab-4011-b2e1-e307b6f2fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.022759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2cbeb-b8ab-4011-b2e1-e307b6f2fbd8 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.024379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 369/736] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:48.025363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2cbeb-b8ab-4011-b2e1-e307b6f2fbd8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.035456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.103691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 368/737] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:48.103691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.142568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2cbeb-b8ab-4011-b2e1-e307b6f2fbd8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.181530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.182184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.182474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.182752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.192003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.193393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-340f407f-a761-44c2-98a2-92c533aeaa02 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:48.206675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:48.207725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:08:48.208678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:48.212170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 370/738] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:48.213655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.214012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.214298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.236442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:48.237655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:48.256797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d871f-0ca3-42dd-aa05-189ad0955142 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.264386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.306945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.308091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.308529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.309265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.314494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 371/739] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:48.315658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.323519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.325771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.328963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:48.343972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:48.345229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:48.346210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:48.346210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:48.346210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:48.351837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 369/740] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:48.353439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.473782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c334ba-8062-46dc-8014-6ddfb690f4ee None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.479345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faefdf-322b-4ead-b6e9-56c85b9e75f2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.511831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.512534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.512870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.513147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.513936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:48.514245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.515721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07085f69-4965-4eea-b431-1c3b25592eb6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.523341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.523988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.524557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.524840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.526588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:48.526904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.528338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 372/741] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433ebc-e91f-4078-8601-6dc1c32999cc None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433ebc-e91f-4078-8601-6dc1c32999cc None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433ebc-e91f-4078-8601-6dc1c32999cc None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 373/742] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:48.539772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca433ebc-e91f-4078-8601-6dc1c32999cc None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.552178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 370/743] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:48.553490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:48.562625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.619041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.657126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca433ebc-e91f-4078-8601-6dc1c32999cc None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.667156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c312c4-b2f4-4d2c-886e-77cb06b9fdc9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:48.693729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.694347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.694631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.694907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.695567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.696974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8df1d-e8e8-4f7a-bc8f-4c04153bb939 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.704311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:48.704931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.705214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.705487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.773693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:48.775193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:48.801120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 374/744] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:08:48 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:48.802391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b915-285d-4b08-b892-6fb59147281e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:48.802703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b915-285d-4b08-b892-6fb59147281e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:48.802982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b915-285d-4b08-b892-6fb59147281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:48.837138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:48.879109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f900b915-285d-4b08-b892-6fb59147281e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:48.879109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f900b915-285d-4b08-b892-6fb59147281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:48.908221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 371/745] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 19:08:48.908221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.004761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 375/746] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 19:08:49.005727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900b915-285d-4b08-b892-6fb59147281e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.008480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.034503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15d17-0cfb-40fc-9606-5aca0499e495 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.087924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.088572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.088852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.089129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.098399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.103731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:49.114485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:49.115582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:49.116511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:49.118934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 372/747] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:49.120014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.137713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900b915-285d-4b08-b892-6fb59147281e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.143700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.187114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.188120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.188579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.189064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.202737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:49.208450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:49.209617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:49.210719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:49.217256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 376/748] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:49.217256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.267472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed83a8b-7d31-4ee3-924c-84e0f63f18ed None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.288184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.314361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.315000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.315279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.315551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.332356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.334619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.336575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31bb8f-3da1-451a-9154-a4ddc87e4218 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:49.353063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:49.354081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:49.360635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:49.360635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:49.360635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:49.360635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 373/749] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:49.360635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.376970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.377584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.377872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.378151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.379682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:49.379682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.380619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.393984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 377/750] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:49.394807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.435721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.479020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fa67a-6bc9-4612-b2bf-bab96245619c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.516004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4e931-e786-4db7-89d9-16e7bba4b849 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.519973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.520931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.521233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.521516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.522186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:49.522501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.523908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.537892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 374/751] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:49.537892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.547626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.561298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.563191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.563191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.563191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.563419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.564949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.639189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 378/752] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:08:49 2022] POST /identity/v3/services => generated 240 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:08:49.644019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.644104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7e8bc-fdd6-4b2e-900f-6e1a8eea5527 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.679170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.698189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.698808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.699412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.699620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.700308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:49.700617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.702068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.716741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 375/753] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:49.717650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.793995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6beb-2736-4f88-95e3-014505b7b0b0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.840993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.841665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.841851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.842185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.843094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=e0c4e352d9914c3a9a5225439274d91d) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:49.843418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.844890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.851142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 379/754] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/role_assignments?user.id=e0c4e352d9914c3a9a5225439274d91d&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:49.851950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.860411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3f723-93b0-4144-b7d0-373d69ebf7b4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:49.902552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:49.903780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:49.904850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:49.905376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:49.907366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:49.931709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:49.934407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:49.953129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:49.954366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:49.955341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:49.955341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:49.955341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:49.957347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 376/755] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:49.963540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:49.984586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c56e-c6be-4a1c-8be5-a87721da3de5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.032282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:50.039021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.039696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.039989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.040266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.050122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.051938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:50.071483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:50.072468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:50.073431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 380/756] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:50.079555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.117836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c52a-8222-43b4-87e8-d70dbd78f534 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.164528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.165169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.165450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.165722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.166374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:50.166678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.168099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.185746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 377/757] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:50.191141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.200287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2da77-92f1-4404-8286-4fecfb44f275 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.207905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:50.239478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.240366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.240793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.241295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.242124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:50.242575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.244741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.258819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 381/758] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:50.266460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.311240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:50.315858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd479c7-e8b3-4248-8fe2-907c535997cb None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.366665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.366952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.367229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.376262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.378094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:50.393778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:50.394742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:50.395746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:50.397168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 378/759] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:50.404855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.424185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e930ced-6f45-4e48-b93b-323da80e7056 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.490944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.491972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.492452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.493367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.495392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:50.495948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.499400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.518120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 382/760] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/role_assignments?user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:50.576148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fa50-19e7-4bfa-80d4-34af2719c1cc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.614287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.615253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.615835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.616310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.617295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:50.617842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.619590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.640844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 379/761] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:50.646801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.683569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:50.769394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ff662-7bbe-4517-873f-c03cd5c6a09a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:50.818814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:50.819456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:50.819764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:50.820042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:50.831133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:50.832628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:50.849315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:50.850727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:50.851668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:50.851668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:50.851668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:50.851668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 383/762] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:50.856356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:50.880936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:50.971730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf917a6-8ec2-4146-9068-8747a10ee15e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:51.009865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:51.010480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.010758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.011030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.011689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:51.012001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:51.014159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:51.026295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 380/763] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:51.032875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:51.160007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:51.187428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d4078-6e08-412e-b9ee-953090ce4163 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:51.235083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:51.235762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.236049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.236341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.250307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:51.251938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3eba2dbe-7a1c-4be4-bcef-1e6cdd14f8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:51.267337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:51.268455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:51.269351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 384/764] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:51.276017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:51.367265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b0958-8d38-49a2-ae64-101b12f0aa48 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.367586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b0958-8d38-49a2-ae64-101b12f0aa48 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.367872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b0958-8d38-49a2-ae64-101b12f0aa48 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.369332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 385/765] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:51.376086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.376373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.376647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.399019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:51.412689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:51.414231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:51.427512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edeec-4015-46e6-bc94-278bad3835fa None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:51.463503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:51.464448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.464926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.465391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.466313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:51.467361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:51.469030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:51.482886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 381/766] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:51.489175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:51.596253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff0183-9ed3-4a32-9b90-71f9ba4dcd1e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:51.605621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 386/767] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:51.634332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:51.635375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:51.635894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:51.636383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:51.639207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=c9b27eb952a04c46bb36653fb79abe3e, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:51.640911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:51.644423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:51.781746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 382/768] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:51 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:52.010560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:52.089581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:52.125929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c2974-32eb-44d4-998e-130d87d44be2 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:52.173302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:52.173946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.174233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.174506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.175094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:52.176481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:52.190528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:52.191533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:52.192408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:52.194941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 387/769] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:52.199595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:52.331505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020b5f-1af2-4bfb-9355-b56f28b7aff3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:52.381856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:52.382402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.382687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.382963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.383631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:52.383963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:52.385341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc86d3-12b7-458e-a1d1-29272edec165 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:52.395895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 383/770] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:52.405317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:52.548332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6116163-1baf-4a13-9c5d-c6a17ea2ffd2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:52.573612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796202-cc91-4a07-86fd-505b74ced343 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.573612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796202-cc91-4a07-86fd-505b74ced343 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.573612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796202-cc91-4a07-86fd-505b74ced343 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.574616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 384/771] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:52.580107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944cec-d205-420b-8160-b9add7a78629 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.580686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944cec-d205-420b-8160-b9add7a78629 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.581167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944cec-d205-420b-8160-b9add7a78629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.601111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-72944cec-d205-420b-8160-b9add7a78629 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:52.602616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-72944cec-d205-420b-8160-b9add7a78629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:52.613357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:52.613980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.614275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.614551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.615188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:52.615494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:52.617003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:52.625198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:52.634173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 388/772] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:52.641304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:52.671180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:52.683128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 385/773] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:08:52.690734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4191f-a9f0-49e8-bb6d-a05776abe3d7 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.690734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4191f-a9f0-49e8-bb6d-a05776abe3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.690734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4191f-a9f0-49e8-bb6d-a05776abe3d7 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.690734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 386/774] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:52.695071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.695357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.695629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.728397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:52.729599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:52.817153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867b56b-aba4-409f-80ae-09394ac968a3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:52.843513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 387/775] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:08:52.876323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:52.876951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:52.877235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:52.877512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:52.878236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] RBAC: Adding query filter params (interface=public, service_id=c698624520444d3a92db8c5ce71a52ee, region_id=RegionOne) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:52.878548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:52.879945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:52.887875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 389/776] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/endpoints?service_id=c698624520444d3a92db8c5ce71a52ee&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:52.894486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:52.974380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.054168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf83358-ff09-4d67-83ec-c0edccf98c0d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.076594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.101753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.131327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca794da1-4c01-4f4c-9081-4c6add630442 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.149064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.149620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.149930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.150175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.150788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.152265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.164956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 388/777] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:53.170811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.171799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.172277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.172710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.173500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.175149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:53.187344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:53.188580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:53.190204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:53.191828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.191982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 390/778] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:52 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:53.199188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.334859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32b7f8-ebb8-4267-9718-5be6b7430b94 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.366407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfe51e-4efc-48e4-b738-28a417e037f3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.374096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.374738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.375018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.375401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.387695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4b5c9bc7-754c-4388-ba6e-bd1d710c0b7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:53.394205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:53.395212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:53.396222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 389/779] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:53.396920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7f19-2425-4210-8ede-b02ff5d840eb None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.397485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7f19-2425-4210-8ede-b02ff5d840eb None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.397485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7f19-2425-4210-8ede-b02ff5d840eb None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.399438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 390/780] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:53.403379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.406342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.406735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.406911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.420463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.420463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.420845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.421033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.421719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] RBAC: Adding query filter params (name=network) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:53.421949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.423313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.427512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:53.428754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:53.434156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 391/781] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:53.435100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.520401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.522471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 391/782] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:53.524622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.561460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404e1ae-b3cd-4f03-8e4e-7cec0e4f9a06 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.597920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.598880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.599365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.599900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.600789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:53.601295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.602850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.617873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 392/783] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:53.624125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.726005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c939-f2f3-4ed3-99d5-54dbf10ab2dd None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.756050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.778725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ce227-623b-4dce-bf82-2fbff909a7c1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.790029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.790760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.791116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.791467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.792204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] RBAC: Adding query filter params (type=network) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:53.792582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.794220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.808217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 392/784] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:53.813924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.832086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.833031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.833499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.834026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.852733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.854226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.861192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:53.873829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:53.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:53.875743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:53.875743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:53.875743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:53.875743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 393/785] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:53.876385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:53.942895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe781090-5b0a-4b22-8fb9-8ec5fe3d3c58 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.979627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:53.980671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:53.981159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:53.981758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:53.982837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] RBAC: Adding query filter params (interface=public, service_id=0bdccecd5151447f865fd2e6fdf810b1, region_id=RegionOne) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:53.983356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:53.984957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:53.998663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 393/786] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/endpoints?service_id=0bdccecd5151447f865fd2e6fdf810b1&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:53.998828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e0de-68c4-43e6-b7a3-1d103298f7ab None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:53.999992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.006319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.037805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.038437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.038718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.038980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.051260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:54.057381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:54.058334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:54.059208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 394/787] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:54.061101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.118824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6bcc6-dc7f-48c5-b301-abe6074f0414 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.120345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.155130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.156325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.156919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.157392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.158225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:54.158708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.160298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.173983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 394/788] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:54.175173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.178620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8e8c-4df3-4d59-bc1f-af39858b3caa None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.218143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.218751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.219031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.219303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.219932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.221511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.231885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 395/789] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:54.231885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.265896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.365765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06538433-227d-4ace-8cc5-34d78a1272b6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.374501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13137-3f03-470f-a593-a423fd0f3d62 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.508237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.511688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.511688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.511688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.511688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:54.511688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.512302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.524404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 395/790] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:54.528428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.529707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.530802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.530802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.531237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.547588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.549071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.568244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:54.572069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:54.573018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:08:54.574000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:54.581044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 396/791] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:54.586670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.686245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf5cc0-2ab8-4f3b-9f08-dc572ce30adc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.723519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.724179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.724462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.724739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.748070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.748070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.770529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.781443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0980d4-f433-43aa-9419-5b2b819cb3dc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-17af1707-b570-4ab9-afd0-dd91cb13153e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:54.795583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:54.796796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:54.797703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:54.797703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:54.797703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:54.797874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 396/792] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:54.802423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880182-b4e2-4435-a325-6841ec2d42db None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.802423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880182-b4e2-4435-a325-6841ec2d42db None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.802423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880182-b4e2-4435-a325-6841ec2d42db None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.805812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 397/793] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:54.807728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2880182-b4e2-4435-a325-6841ec2d42db None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.821890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.822718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.823097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.823501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.824440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:54.824849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.826685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c188106-1086-48f6-9145-536322bc9906 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:54.852059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 397/794] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:54.853482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.853975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.854360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.875707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:54.877055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:54.896515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:54.934050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2880182-b4e2-4435-a325-6841ec2d42db None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:54.958348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 398/795] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:54.959741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:54.980378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:54.981119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:54.981505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:54.981848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:54.983568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:54.984030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:54.986039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.011242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 398/796] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:55.018959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.087832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.132398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6726348-20f6-4d37-b94a-e3a1733f7120 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.182441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab00f52-11de-4586-9dba-5708fab81605 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.193344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.194009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.194263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.194538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.212196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.213714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:55.239385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:55.240413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:55.241679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:55.241679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:55.241679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:55.241679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 399/797] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:55.249041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.250895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.251541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.251848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.252121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.257619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.262863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.264969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:55.277140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:55.287750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 399/798] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:55.288718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.391084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b0b2-2935-4698-8382-3dfe86fe092d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.421232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.423474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11000a-e5d5-413c-b132-766dadb0583f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.432815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.434247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.434890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.437460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.438482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:55.438901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.440497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.456855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 400/799] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:55.458170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.461935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.462799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.463267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.463771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.464820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:55.465308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.467494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.489171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 400/800] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:55.489171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.548131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.570333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd51c-421c-4d8d-b070-f33fd7b52798 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.597150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0df9-1f06-48a4-a668-31d8b0d686cb None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.608600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.609559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.609994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.610416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.623778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:55.628964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:55.630158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:55.631202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:55.633563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 401/801] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:55.634586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.645597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154384b-30b7-4919-9296-279ce614278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.646170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154384b-30b7-4919-9296-279ce614278c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.646461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154384b-30b7-4919-9296-279ce614278c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.646741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154384b-30b7-4919-9296-279ce614278c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.653932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154384b-30b7-4919-9296-279ce614278c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.655384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154384b-30b7-4919-9296-279ce614278c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-0154384b-30b7-4919-9296-279ce614278c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:55.667185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:55.668332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:55.669287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:55.671966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 401/802] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:55.672808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154384b-30b7-4919-9296-279ce614278c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.704020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.748569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e0a7-d23d-4ecb-ada8-4b3580d4be44 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.783745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154384b-30b7-4919-9296-279ce614278c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.785671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981697-223f-4594-bb0b-11912f43876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.786552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981697-223f-4594-bb0b-11912f43876b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.786979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981697-223f-4594-bb0b-11912f43876b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.787399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981697-223f-4594-bb0b-11912f43876b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.798463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.800524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55981697-223f-4594-bb0b-11912f43876b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.802125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55981697-223f-4594-bb0b-11912f43876b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-55981697-223f-4594-bb0b-11912f43876b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:55.818749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:55.819899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:55.820812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:55.820812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:55.820812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:55.820812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 402/803] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:55.820812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981697-223f-4594-bb0b-11912f43876b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.821355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.821978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.822293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.822540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.823199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:55.823503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.825129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbef1-ada6-4633-994b-b4aaac1b899c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.835382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 402/804] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:55.837202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760a03e-a346-48ff-ae35-50112068cd7f None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.837553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760a03e-a346-48ff-ae35-50112068cd7f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.837895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760a03e-a346-48ff-ae35-50112068cd7f None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.840086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 403/805] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:55.840923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760a03e-a346-48ff-ae35-50112068cd7f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:55.941000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981697-223f-4594-bb0b-11912f43876b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.943272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:55.950824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760a03e-a346-48ff-ae35-50112068cd7f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:55.981568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.982716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.983071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.983340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.983989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:55.984299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.985720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840213a-3335-4db8-8bcd-b73309cc7c41 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:55.987157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:55.987670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:55.987965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:55.988239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:55.988870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:55.989176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:55.990515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.003732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 404/806] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:56.004101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.004478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 403/807] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:56.005857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.006155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.006445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.024977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:56.026164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:56.043998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.103824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 404/808] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:56.103824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.118583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86866-1397-410b-a1b0-ea13d614c67e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.156611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.157256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.157544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.157824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.168654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.170142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:56.187017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:56.188213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:56.189336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:56.189336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:56.189336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:56.193489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 405/809] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:56.194362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.214989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.225863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f10b-c550-44a7-b18f-562f5c0e9732 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.269774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.270733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.271179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.271607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/e0c4e352d9914c3a9a5225439274d91d/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.274615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=e0c4e352d9914c3a9a5225439274d91d, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.276905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.297518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.309660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd2db-d70a-4c50-b5f3-ead87f7d3e69 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.345551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.346689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.347236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.347819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.355970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 405/810] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:56 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/e0c4e352d9914c3a9a5225439274d91d/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:56.357164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.365948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.368586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:56.385596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:56.396350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 406/811] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:56.397313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.503961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce8a02-ccbf-4bfb-a865-a90374ef9d5e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.555039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.556003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.556445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.556870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.557552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:56.557863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.559323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.564206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c6436-c62d-4108-80ab-d2e76fb0a464 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.573752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 406/812] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:56.574437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.591787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.611022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.611994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.612433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.612832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.613604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:56.614062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.616183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.633338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 407/813] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:56.634738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.649812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.703389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7913-389b-4352-b061-aa21e768e991 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.743561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.744241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.744523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.744798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.745391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.746764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:56.756669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:56.757872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:56.758739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 407/814] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:56.759536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:56.844494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:56.868317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ca54e-57b8-4c2b-954e-c16661c14bb6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.869096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e665d1-051e-4fa7-af8d-0c1743bc0011 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:56.907411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.908085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.911190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.911190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.914205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:56.914967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.915503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.916039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/a2a29c5a3e2a43bb94c0bd21b33c9ff8/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.916418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.917879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a3c2642a-3d47-48c7-af99-ebf75a43ba7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:08:56.930758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:56.931883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:08:56.933032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:56.936084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=a2a29c5a3e2a43bb94c0bd21b33c9ff8, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:56.937331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 408/815] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:56.937787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:56.938448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81fb75-55d4-4e1d-ae0c-17e968c2cdb8 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:56.938733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81fb75-55d4-4e1d-ae0c-17e968c2cdb8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:56.939009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81fb75-55d4-4e1d-ae0c-17e968c2cdb8 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:56.940539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 409/816] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:56.940977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81fb75-55d4-4e1d-ae0c-17e968c2cdb8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.009675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.047678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 408/817] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:08:56 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/a2a29c5a3e2a43bb94c0bd21b33c9ff8/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:08:57.047678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.056197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81fb75-55d4-4e1d-ae0c-17e968c2cdb8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.110375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.111203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.111543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.111866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.112641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:57.112945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.114453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a654e9-0303-48cd-a62f-700692503475 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.120371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 410/818] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:57.121668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.121953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.122227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.143317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:57.144682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:57.198406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930db69-8c61-405a-98d5-595998cb2a8c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.230871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 411/819] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:57.231918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.243501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.251577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.251942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.252220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.252496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.253305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:57.253602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.255007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.276394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 409/820] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:57.283131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.307438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.356016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265802a2-9ee3-4c39-ae5f-7aa956f1e59b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.409550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.410338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.410663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.411105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.411895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:57.412196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.413634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.420976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7369-2249-45e6-842a-e32cc82be842 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.424479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 412/821] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:57.429897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.465709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.466887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.467359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.467933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.538142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.554580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810c31-0fdb-4cc6-be28-57777589ae62 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.558500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.560304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.599439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.610629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.613585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.613585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.613585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.613585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.614013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.680607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 410/822] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:08:57.681854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.743979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 413/823] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:08:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:08:57.745467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.800218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:57.848279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c937ed-c64c-4aaf-b79e-f982734aabba None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:57.908752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:57.909833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:57.910332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:57.910805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] PATH_INFO: `/v3/services/network` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:57.911738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:57.913317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:57.923447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:57.924529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 19:08:57.925349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:08:57.928358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 411/824] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:57.944718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:57.969451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24203891-43cc-46c4-8031-60a70d423aa6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.011760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.012397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.012675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.012949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.025141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.026362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:58.042099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:58.043114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:58.044056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:58.044056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:58.044056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:58.044056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 414/825] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:58.049664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.074907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395fe0f-87a2-4076-ac5d-bbe984515778 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.088020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:58.116335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.117315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.117827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.118325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.119163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:58.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.123351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.136520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 412/826] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:58.143925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.168651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:58.207930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1f866-63fa-4cf1-983e-383abd4fbacc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.296383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.297349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.297833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.298302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.298687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53950-7b2f-4b04-ae5c-51759d3da5f1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.299683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:58.300175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.301761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.323033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 415/827] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:58.328726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.338673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.339275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.339556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.339852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.340479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] RBAC: Adding query filter params (type=network) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:58.340781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.342170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.356635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 413/828] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:08:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:58.360965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.479487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:58.511957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e08a63-47ad-4191-8618-6db857ce4f2e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.527579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:58.550429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da20a3-ba28-4179-8234-8a09ea9c494a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.551357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.551996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.552278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.552548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.553219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.554686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283ad1f-b3cb-49b7-bdd1-417421d8536d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.596170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.596721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.597002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.597273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.604437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.606007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:08:58.619813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:58.620872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:58.621764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 416/829] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:08:58.627809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.643110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 414/830] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:08:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:08:58.761693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b63a2-e82d-49fb-abbd-78c639d75883 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.762586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b63a2-e82d-49fb-abbd-78c639d75883 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.762961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b63a2-e82d-49fb-abbd-78c639d75883 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.765927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 415/831] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:08:58.771808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.772099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.772362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.797481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599599-1b00-4dba-acb2-b1c6546a092e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:58.807676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:58.807676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:58.877019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:58.877983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:58.878473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:58.878948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:58.879868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:58.880399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:58.882040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:58.883952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 416/832] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:08:58.895343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 417/833] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:58.904191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:58.976048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:59.009686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6190f-39c9-4ddc-a9f8-71d69832e378 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:59.045591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:59.046198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.046477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.046749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.047535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:59.047914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:59.049337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:59.055702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 417/834] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:08:58 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:59.187188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:59.202187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:59.301784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11288b7-c41e-42d1-aa4d-50949c4c5fd7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:59.340059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:59.340692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.340969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.341861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.352653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:08:59.357556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:59.358514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:59.359374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 418/835] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:59.364882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:59.470160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a026c4-52bb-43ff-b376-23422316d206 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:59.493089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:59.504757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:59.505380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.505660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.505933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.506654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:59.506958 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:59.508398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:59.518607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 418/836] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:08:59.524439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:59.600232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:59.635430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d82c4-49c3-42c3-bfb7-eee93b2e6675 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:59.672978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:59.673795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.674073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.674341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.685420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:59.686860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4417ca44-c82e-4364-968a-b7ffbb981d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:08:59.701692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:08:59.703938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:08:59.705623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:08:59.705623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:08:59.705623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:08:59.705623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 419/837] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:08:59.708497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:59.723792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5007-d04e-4b46-8432-6cd2b9f02a82 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.723792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5007-d04e-4b46-8432-6cd2b9f02a82 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.723792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5007-d04e-4b46-8432-6cd2b9f02a82 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.723792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 420/838] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:08:59.729191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.729479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.729753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.749013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:08:59.750214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:08:59.827675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 421/839] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:08:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:08:59.827905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e2ff6-95f7-4c38-b44b-02a692fc91c2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:08:59.862203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:08:59.862815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:08:59.863099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:08:59.863372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:08:59.864082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:08:59.864389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:08:59.865806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:08:59.878891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 419/840] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:08:59.884291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:08:59.942314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:08:59.991458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb588c-f4d7-468b-9560-d4f9ca7ad538 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.027474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.028143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.028426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.028747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.036650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.038099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:00.050324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:00.051860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:00.053420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 422/841] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:08:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:00.062108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.115385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.136156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.173635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b4306-6a4a-4c81-9d91-356aecc5f49b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.214576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.215569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.216058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.217072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.217989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:00.218496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.220335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.245550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 420/842] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:00.246749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.281911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8febda-75e4-4955-bdc5-b20edf6c384a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.297975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.321515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.322219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.322856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.323597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.335636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-f2f3f984-37c3-43ec-9376-9f4ac24ba14b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:00.340861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:00.342790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 423/843] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:00.344174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d0a0a-2d4f-41cd-9b2e-9423e39995d4 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.345381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d0a0a-2d4f-41cd-9b2e-9423e39995d4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.345381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d0a0a-2d4f-41cd-9b2e-9423e39995d4 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.347771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 424/844] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:00.349713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d0a0a-2d4f-41cd-9b2e-9423e39995d4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.360161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b3f17-9792-415d-9ea5-15d6d0e5ada6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.361849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.399039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.399814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.400010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.400503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.459698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d0a0a-2d4f-41cd-9b2e-9423e39995d4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.473327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.474768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.495120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.495810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.496038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.496616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.497005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:00.497311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.499177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7517c5-2a14-4688-bec0-93670b6f65fe None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.512173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 425/845] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:00.512173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.512173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.512173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.527990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:00.529197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:00.561132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.588808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 421/846] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:00.589760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.592784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.607968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 426/847] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:00.609541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.704122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895b71-6b19-4fb8-9d80-366ce0de14f2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.729092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29cb63-c2da-4be3-8f3c-9be10cefdec6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.754189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.755376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.756152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.757039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.768699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.769880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.770279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.770567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.771039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.771794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.784647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.786433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:00.786882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:00.787840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:00.788760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:00.788760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:00.788760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:00.788760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 422/848] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:00.796063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:00.808718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:00.809758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:00.810695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:00.810695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:00.810695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:00.815777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 427/849] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:00.818250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.845483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.862768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:00.918250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787d24f-4737-4324-b894-887d2ed14e0e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.943282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dbe29-1bfb-4212-986d-2da369406b51 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:00.955204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.955839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.956120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.956391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.957071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:00.957354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.958704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:00.975712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 423/850] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:00.975712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:00.985840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:00.986449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:00.986730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:00.987002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:00.987689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:00.987992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:00.989346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.001492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 428/851] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:01.004746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.079440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615ec83-2eb9-47c8-83e8-bcc5f0fc46fe None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.089781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.096196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.112445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a283-3bb6-472f-9ea0-72667179e5f3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.115587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.116243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.116529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.116834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.117377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.118754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:01.128343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:01.129994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:01.131042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:01.134428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 424/852] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:01.135320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.153804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.154641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.154918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.155203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.163249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.164823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:01.176958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:01.177919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:01.178849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 429/853] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:01.179470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.246145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c705b8-030d-4190-a9f7-59086bd9ae98 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.282591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe7d3-ab1d-4886-b6a4-6cbe5b5eb607 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.286214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.286903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.287193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.287475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.298140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.299788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:01.316911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:01.317892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:01.318803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 425/854] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:01.319286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.326762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.327380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.327678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.327957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.328616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:01.328921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.330360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.333894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.337153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 430/855] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:01.337760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.369934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.432960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3eaf7-27af-4de8-b733-8647ea8e4562 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.441021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21404e-64b3-4539-83bc-c004d8911e7b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.465913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.467038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.467323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.467603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.468356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:01.468668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.470082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.473888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.474632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.474916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.475188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.475904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:01.476208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.477611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6911-a7b5-41d9-8d49-fe305b87fb73 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.488221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 426/856] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:01.489032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.491929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 431/857] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:01.493068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf031-678b-435a-b0b2-f08d74a0c29f None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.493358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf031-678b-435a-b0b2-f08d74a0c29f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.493632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf031-678b-435a-b0b2-f08d74a0c29f None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.494862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 432/858] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:01.495558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbf031-678b-435a-b0b2-f08d74a0c29f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.554273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.583136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.589116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a756ddd-5fec-49e7-a9a6-68e59da5c29e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.595555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbf031-678b-435a-b0b2-f08d74a0c29f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.623093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.623730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.624013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.624284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.624902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:01.625203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.626708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.628605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.629192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.629467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.629737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.631151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 427/859] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:01.631968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.707997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.709465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.760003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0c5c5-4fc0-485f-bbbb-a6898a9bc5a9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:01.775239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.799378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:01.800031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.800318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.800593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.801316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=e0c4e352d9914c3a9a5225439274d91d) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:01.801627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:01.803039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617ae1a-d0cc-4b75-9bde-28840c3f02d8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:01.809526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 428/860] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/role_assignments?user.id=e0c4e352d9914c3a9a5225439274d91d&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:01.810508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:01.810794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:01.811068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:01.832001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 433/861] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:01.835458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:01.836776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:01.842427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:01.860465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:01.925170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 429/862] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:01.978442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be607a-3a30-4ea9-903b-d0af5fa8628d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.017382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.018431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.018959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.019427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.030833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.032625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:02.048397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:02.049351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:02.050248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:02.050248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:02.050248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:02.050419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 434/863] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:02.056542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.191572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f682ac-648f-4a20-b397-0f858eabe418 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.196156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.212389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:02.229417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.230075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.230363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.230755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.231438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:02.231809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.233317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.249995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 430/864] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:02.251236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.268777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:02.306414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3ca6d-5dc4-4f98-88de-7dc1caf4186a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.344034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.344843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.345227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.345604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.346296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.347811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:02.360616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:02.361742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:02.362649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:02.363433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 435/865] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:02.364417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba92b-7556-4b9f-8fc1-9eb0549d30d8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.368555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.406690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.407996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.408495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.408956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.416753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.418684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:02.432784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:02.434182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:02.435692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:02.438119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 431/866] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:02.444226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.475085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:02.505923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057b4c-aa1f-40dd-92b6-568f18c8fd17 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.531475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:02.556308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.556950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.557234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.557511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.558448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:02.558760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.560978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.564763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47c6b-8a21-4be0-b63c-911b5c67a494 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.566934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 436/867] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:02.572045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.608308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.609392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.609960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.610473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.611392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:02.611943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.613900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.628620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 432/868] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:02.634434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:02.688252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b3591-14eb-44e2-8553-ef016a3baddf None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.725314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.725850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.726134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.726413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.727203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:02.727538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.729043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbf2e-3194-45db-b034-4b11f4d561fc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.734672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 437/869] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:02.753609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a6c14-7390-4075-819f-2bfc309467b5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:02.793721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:02.795003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:02.795324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:02.795607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:02.796381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:02.796695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:02.798264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb6854-ae5a-40f8-bd43-b9382e7f07c5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:02.800418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:02.811024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 433/870] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:02 2022] GET /identity/v3/role_assignments?user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:03.870915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdbfc4-5126-451c-ad8d-00eb8abe3869 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:03.871557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdbfc4-5126-451c-ad8d-00eb8abe3869 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:03.871557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdbfc4-5126-451c-ad8d-00eb8abe3869 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:03.873638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 438/871] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:03.878864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52025bb-61f0-4b53-8393-5aa7561d98ba None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:03.879281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52025bb-61f0-4b53-8393-5aa7561d98ba None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:03.879470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52025bb-61f0-4b53-8393-5aa7561d98ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:03.897459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a52025bb-61f0-4b53-8393-5aa7561d98ba None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:03.898639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a52025bb-61f0-4b53-8393-5aa7561d98ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:03.969874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 434/872] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:04.145759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4353dc-375f-441d-ae04-f92710bfb0cf None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.145954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4353dc-375f-441d-ae04-f92710bfb0cf None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.146226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4353dc-375f-441d-ae04-f92710bfb0cf None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.147820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 439/873] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:04.153284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.154101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.154380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.173711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:04.174893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:04.239808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4353dc-375f-441d-ae04-f92710bfb0cf None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:04.243853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:04.251783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 435/874] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:04.346714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4353dc-375f-441d-ae04-f92710bfb0cf None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:04.382535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:04.383147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.383431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.383731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.384437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:04.385863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:04.450450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 440/875] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:04 2022] POST /identity/v3/services => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:04.515847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:04.583938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:04.630295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8d81b-7e54-4a2c-b074-0f5adcfa489a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:04.674977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:04.675617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.675964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.676244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.676899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:04.678271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:04.688543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:04.689677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:04.690609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 436/876] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:04.695810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:04.763696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:04.805202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208354fc-aadc-4504-a4c8-7422bb6dcbae None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:04.840911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:04.842087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.842424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.842704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.843402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:04.843739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:04.845192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fd8e4-e85c-4862-9faf-5eb423741ccb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:04.855784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 441/877] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:04.857779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:04.971121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b805-5792-4c76-af0a-647092be9046 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:04.978598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1414ec-edc3-4b3c-af17-b139bd4124a2 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.979122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1414ec-edc3-4b3c-af17-b139bd4124a2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.979122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1414ec-edc3-4b3c-af17-b139bd4124a2 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.980782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 442/878] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:04.988044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cbd4-1a19-4c2f-b2c6-3a0ce52a362f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:04.988333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cbd4-1a19-4c2f-b2c6-3a0ce52a362f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:04.988615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cbd4-1a19-4c2f-b2c6-3a0ce52a362f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:04.992658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:05.010601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6544cbd4-1a19-4c2f-b2c6-3a0ce52a362f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:05.012993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6544cbd4-1a19-4c2f-b2c6-3a0ce52a362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:05.016728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:05.017298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.017585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.017943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.018585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:05.018896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:05.020893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a7a9-97a4-4b26-b5a7-3a5242128d05 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:05.027637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 437/879] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:05.095590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 443/880] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:05.213839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fea9-6fc7-408f-b9f3-9c2fdbf401bb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.214150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fea9-6fc7-408f-b9f3-9c2fdbf401bb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.214395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fea9-6fc7-408f-b9f3-9c2fdbf401bb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.216097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 438/881] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:05.221229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.221525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.221820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.241046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:05.242236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:05.299428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:05.314816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 444/882] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:05.387377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fea9-6fc7-408f-b9f3-9c2fdbf401bb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:05.502134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fea9-6fc7-408f-b9f3-9c2fdbf401bb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:05.538361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:05.538983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.539269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.539546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.540247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:05.541600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:05.584862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:05.637970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 439/883] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:05 2022] POST /identity/v3/services => generated 269 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:05.713762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baeb54-3d06-49d7-905d-53b6c7cd088e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:05.762969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:05.764494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.764980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.765472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.779638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:05.785388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:05.786451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:05.787304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:05.794928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 445/884] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:05.797528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:05.848409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:05.912103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0537d8-1f16-4749-a7e3-554604748d83 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:05.950142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:05.951818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:05.952324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:05.952841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:05.953757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:05.954288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:05.955945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:05.970216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 440/885] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:05.972531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:06.024083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:06.080309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d275-8ce5-43d5-a4c5-7df45142a0d1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:06.118140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:06.119100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.119606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.120113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.131762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:06.133544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:06.149484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:06.150563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:06.151562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:06.151562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:06.151562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:06.158393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 446/886] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:06.160030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:06.267437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553a4bb-edc1-4969-8da5-5d739962e7d5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:06.270218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:06.305081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:06.305697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.305975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.306249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.306890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:06.307194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:06.308608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:06.323666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 441/887] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:06.329384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:06.431275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:06.438126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec91e71-f3e8-495f-9643-fa7de9e901ac None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:06.473227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:06.474276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.474768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.475256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.484445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:06.486164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:06.498167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:06.499921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:06.500895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:06.507753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 447/888] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:09:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:06.509412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:06.613566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cf27-b31f-4d8a-8a6f-4b1c2e41b223 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:06.653283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:06.654444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.654731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.655008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.656069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:06.656379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:06.657872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c0aee-c02e-4366-84b3-4f07d5093c15 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:06.679612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 442/889] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:09:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:06.685604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:06.762482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccd42-8fe7-41ff-a652-44d78f32f15e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.763335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccd42-8fe7-41ff-a652-44d78f32f15e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.763556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccd42-8fe7-41ff-a652-44d78f32f15e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.765157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 443/890] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:06.770287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.770566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.770839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.792680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:06.794336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:06.796571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba408-def4-4cbc-9f84-40e0b3f33fae None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:06.820065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:06.831784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:06.832528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:06.832911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:06.833361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:06.879807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 444/891] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:06.880071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:06.921776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:06.928563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.036261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 448/892] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:06 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:07.042618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.145508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7277d-89eb-4b0f-8537-231d1208eb11 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.187246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.187952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.189142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.189649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.190122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.200864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.202619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:07.218277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:07.219287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:07.220590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:07.220590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:07.220590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:07.226758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 445/893] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:07.228407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.233784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.256940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.301444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695112a-ec2a-4e10-88a2-4e7b7a049ab5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.335598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531ad3-dd6a-46c6-bb83-22bb189b999c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.340018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.340654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.341425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.341700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.342285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.344151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-5a8bdc04-0261-4a84-a678-e5e1f52a1b4a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:07.353844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:07.359906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:07.360824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 449/894] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:07.361630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0c7f3-3671-4be0-b30e-a3db0aa95c05 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.361630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0c7f3-3671-4be0-b30e-a3db0aa95c05 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.361630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0c7f3-3671-4be0-b30e-a3db0aa95c05 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.361630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 450/895] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:07.362407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0c7f3-3671-4be0-b30e-a3db0aa95c05 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.378624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.379547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.380005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.380429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.381346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:07.381793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.383403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bdd5e-fa30-42df-aa3e-28864802de74 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.400846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 446/896] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:07.400846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.400846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.400846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.418306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:07.420152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:07.471226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0c7f3-3671-4be0-b30e-a3db0aa95c05 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.483465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.493505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 447/897] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:07.495196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.508820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.513832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.514453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.514782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.515062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.515809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:07.516173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.517661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.527349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 451/898] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:07.529167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.630038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13dfbd-94bf-4941-936f-eb5389d8405a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.631789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84a1e8-b39d-4f67-9444-190ddb51bae7 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.670679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.671421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.671729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.672006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.672632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:07.672939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.674381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.675943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.676683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.676973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.677252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.684277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 452/899] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:07.684425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.685706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.686323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:07.698855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:07.699829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:07.701281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 448/900] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:07.708064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.742979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.775256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.798941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a125e09-3e6d-40f0-8255-ed98312c819d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.828126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29929d-04ef-4d14-8986-a1104bc7f53f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.835367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.836421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.836710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.836991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.837698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] RBAC: Adding query filter params (interface=public, service_id=cde5b28739434219ae9feab4650b549b, region_id=RegionOne) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:07.838012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.839773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.845814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 453/901] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/endpoints?service_id=cde5b28739434219ae9feab4650b549b&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:07.846643 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.873100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.874159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.874628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.875082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.876004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:07.876753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.878537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:07.897738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 449/902] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:07.898910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:07.957097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d54df-7b89-44a9-80e3-50cb98f84e07 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:07.971838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:07.993033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:07.993639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:07.993921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:07.994195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:07.994844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:07.996249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4a20f-e200-47f4-8184-f5ad2182d87d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.006869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640c49-96e5-4bb9-9704-cf0db6e39f16 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:08.020373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:08.040434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:08.041038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.041323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.041601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.042203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:08.043577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.050214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 450/903] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:07 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:09:08.050480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:08.078837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 454/904] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:07 2022] POST /identity/v3/services => generated 241 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:08.080837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b2e0-4564-4c0d-b2d7-cf9a0b761232 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.081133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b2e0-4564-4c0d-b2d7-cf9a0b761232 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.081584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b2e0-4564-4c0d-b2d7-cf9a0b761232 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.083224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 455/905] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:08.089441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.089728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.091323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.111756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:08.113033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:08.196923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e509-f7b1-48f2-b59e-dc3f6972d0dd None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:08.213815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 456/906] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:08.245408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:08.246095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.246380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.246658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.248031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] RBAC: Adding query filter params (role.id=1dcd4d4337894b4ca66eaf9110717d66, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:08.248343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:08.249858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.256745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 451/907] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:08 2022] GET /identity/v3/role_assignments?user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1dcd4d4337894b4ca66eaf9110717d66 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:08.496165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:08.597759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:08.602307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636d71-d0dc-41a7-be61-1aa464888c4b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:08.638482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:08.639170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.639456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.639753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.640389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:08.641824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:08.650805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:08.651818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:08.652816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 457/908] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 15 19:09:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:08.661383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:08.707371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:08.766667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650fce-e2a8-4d21-9399-dfe6732c549d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:08.803833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:08.804825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.805338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.805817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.806660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:08.807211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:08.809203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.815136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 452/909] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:08.821229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:08.926353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b864c-1221-4f7b-91c8-296368b121b5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:08.961501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:08.962114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:08.962400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:08.962679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:08.963317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:08.963626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:08.965030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:08.970993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 458/910] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:08.977068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:09.084957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c3451-051c-4d2e-937e-747f2e7137ac None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:09.092711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:09.120206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:09.120816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:09.121101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:09.121437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:09.122169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] RBAC: Adding query filter params (interface=public, service_id=b679ca3e1c714ce7834b3fe048c9d507, region_id=RegionOne) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:09.122482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:09.123862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddd8b2-da9e-4176-b2ac-9b7edf1bb799 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:09.129198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 453/911] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:09:08 2022] GET /identity/v3/endpoints?service_id=b679ca3e1c714ce7834b3fe048c9d507&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:09.134988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:09.165145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:09.242538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dbd28-8b74-4816-be17-90b0b15f48a6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:09.276699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:09.277306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:09.277593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:09.277871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:09.279034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:09.280487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2222a2d-8e8a-4b3f-a6f6-401fb2a7f31f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:09.287011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 459/912] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:09 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:09:10.299163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd16ae-1f97-41f3-85a8-7dbc794b7fc3 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.299721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd16ae-1f97-41f3-85a8-7dbc794b7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.299721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd16ae-1f97-41f3-85a8-7dbc794b7fc3 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.301316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 454/913] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:10.306853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073431e-5d2f-4f03-b8be-8c4603dd28f2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.307268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073431e-5d2f-4f03-b8be-8c4603dd28f2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.307458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073431e-5d2f-4f03-b8be-8c4603dd28f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.326200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0073431e-5d2f-4f03-b8be-8c4603dd28f2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:10.327403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0073431e-5d2f-4f03-b8be-8c4603dd28f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:10.349319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a7ce-7563-4c93-bce2-4625530556bf None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.349625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a7ce-7563-4c93-bce2-4625530556bf None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.349883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a7ce-7563-4c93-bce2-4625530556bf None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.351622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 455/914] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:10.357573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8691a0-e6ab-4a1b-a690-bad824dd7163 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.357924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8691a0-e6ab-4a1b-a690-bad824dd7163 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.358208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8691a0-e6ab-4a1b-a690-bad824dd7163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.377170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0e8691a0-e6ab-4a1b-a690-bad824dd7163 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:10.378477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0e8691a0-e6ab-4a1b-a690-bad824dd7163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:10.409870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 460/915] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:10.455909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 456/916] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:10.497200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff03ee-a5b5-440e-b126-ef0fb749ce74 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.497502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff03ee-a5b5-440e-b126-ef0fb749ce74 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.497771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff03ee-a5b5-440e-b126-ef0fb749ce74 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.499109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 461/917] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:10.504345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.504630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.504903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.523335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:10.524553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:10.597738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 457/918] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:10.683087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff03ee-a5b5-440e-b126-ef0fb749ce74 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:10.705923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:10.729410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:10.788502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff03ee-a5b5-440e-b126-ef0fb749ce74 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:10.789361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:10.823598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:10.824301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.824591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.824871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.825979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:10.827353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:10.833495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84fcf-4dbd-451d-9fc3-7b261c9ec135 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:10.836564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:10.837546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:10.838414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 462/919] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:10.844387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:10.869523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:10.870130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:10.870414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:10.870688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:10.871279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:10.873739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:10.884120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:10.885934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:10.886988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:10.892160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 458/920] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:10.892160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:10.947467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:10.964072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793664c-f6ee-4f40-81e9-46f6b1c66358 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:10.998754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.001565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.002315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.002456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.002723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.003352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.003656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.005057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.005864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b6018-902d-4089-b7af-34414d6d5590 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.010244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 463/921] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.010953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.047099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.047763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.048049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.048324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.060025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:11.065118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:11.066074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:11.066916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 459/922] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:11.067724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.116724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8b1e-a44f-478b-8022-75180e0a697b None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.150874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.151409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.151721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.152005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.152636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.152946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.154348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.164463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 464/923] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.164463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.179063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.183866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f478a2-18dd-4d3a-9b93-9f2e6ad09daf None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.224787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.225714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.226008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.226535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.227170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.227553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.228861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.239300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 460/924] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.239300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.259012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.273760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd76-482e-449f-8dcc-b8c21add0898 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.311194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.311899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.312136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.312387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.313051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.313361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.314746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.327699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 465/925] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.327699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.352510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a459a45-fffd-4692-ac53-e6cc6106e27b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.392569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.393186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.393472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.393750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.394385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.394691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.396148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.407446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 461/926] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.409174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.414274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.442225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0b3b-9fc7-49d1-90bc-8c1f728c9e54 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.480288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.480897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.481179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.481945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.482616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.484014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.495936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.513724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb34a-d7d5-4594-967f-a0dbc47d1b4f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.546471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.547093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.547380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.547679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.557267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.558697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.568906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 466/927] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:11.570105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2af86e79-cba6-406b-bcd2-a410a7828d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:11.574682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:11.579881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:11.581387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:11.581387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:11.581387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:11.581387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 462/928] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:11.581617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c4a2-eaf4-4f53-98af-8ce080bde915 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.581697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c4a2-eaf4-4f53-98af-8ce080bde915 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.581976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c4a2-eaf4-4f53-98af-8ce080bde915 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.583568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 463/929] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:11.584953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c4a2-eaf4-4f53-98af-8ce080bde915 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.692663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bae72-e8a9-40e3-9be7-9c8b347811d2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.704745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c4a2-eaf4-4f53-98af-8ce080bde915 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.723463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.734224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.734851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.735135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.735402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.736167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] RBAC: Adding query filter params (interface=public, service_id=6e3bdcd93224416db9da05523565e5e4, region_id=RegionOne) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.736512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.737860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2911eba-4cf7-43b7-b163-97f51f1f143d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.743488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 467/930] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/endpoints?service_id=6e3bdcd93224416db9da05523565e5e4&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.746445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.747006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.747832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.748105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.748988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:11.749110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.749209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.749402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.749600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.750672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.752385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:11.764378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 464/931] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:11.765980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.768520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:11.769728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:11.851680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 468/932] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:11.851680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:11.875237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aefe6-0162-4a56-a8ef-694acc86f518 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:11.911170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:11.911845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:11.912183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:11.912463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:11.913070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:11.914488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:11.921203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 465/933] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:09:11.985078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c85e4e-08ac-4207-96f7-eada0da660ee None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.011209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.028373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.029277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.029731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.030171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.039118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.041031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:12.055523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:12.056788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:12.057958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:12.061940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 469/934] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:09:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:12.071267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.143045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.150900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.201532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3480-1297-41e6-8717-a668746b1b1c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.245609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.247086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.247580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.248115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.249667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:12.250181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.252333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.261613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf22a9-5f82-44bc-8778-9cd9f0c822be None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.273149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 466/935] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:12.279854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.298519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.299248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.299623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.299998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.300825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.303326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:12.313431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:12.314937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:12.315849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 470/936] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:12.320616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.380297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.403908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3e5c-df9b-4625-a773-54da2b604361 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.425878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.439111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d06c4-c04e-40f1-b50f-f18d117bf68f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.448582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.449223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.449511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.449794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.462031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.463928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.479069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.479698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.479987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.480262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.480886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:12.481192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.483103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:12.484609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:12.485611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:12.486510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:12.486510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:12.486510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:12.492640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 471/937] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:12.492774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 467/938] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:12.494312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.494530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.620616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf002b56-8194-46aa-95cc-a7f56a6ee042 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.626254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e026ae-caf0-4487-b177-4ef96e739d26 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.638667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.662220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.662831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.663970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.664466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.665307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:12.665809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.667674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.670362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.671602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.672264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.672860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.673688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:12.674398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.675937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.683688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 472/939] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:12.688792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 468/940] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:12.689997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.695626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.731521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:12.801472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de53f9-3827-44b0-b058-e784676177ed None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.820629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365403a-fe96-49ed-bf7a-f1551d3fa119 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:12.838064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.838820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.839485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.839485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.840330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.841860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.862756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:12.863689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:12.864113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:12.864550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:12.873887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:12.873887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:12.886052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:12.887055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:12.888451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:12.889134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 469/941] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:12.893635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:12.963673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 473/942] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:13.010460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:13.024393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462834bf-fa4a-4c7c-bb9e-d9a72814bff7 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:13.075944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:13.076927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:13.077443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:13.078073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:13.078971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:13.079532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:13.081200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d64f0-9341-4906-91b3-3cdc3c162b96 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:13.099722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 470/943] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:13.104592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:13.138855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:13.218322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892c577-cdb4-4fb5-98b0-c28b23ec32f9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:13.254400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:13.255414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:13.255944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:13.256421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/a2a29c5a3e2a43bb94c0bd21b33c9ff8/roles/1dcd4d4337894b4ca66eaf9110717d66` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:13.258809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=a2a29c5a3e2a43bb94c0bd21b33c9ff8, role_id=1dcd4d4337894b4ca66eaf9110717d66)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:13.260468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baac08-9609-47b2-b830-37c24fdfc77b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:13.347440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 474/944] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:09:13 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/a2a29c5a3e2a43bb94c0bd21b33c9ff8/roles/1dcd4d4337894b4ca66eaf9110717d66 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:09:13.819353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244dfcb-490a-4159-8f6e-efce844f698f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:13.819905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244dfcb-490a-4159-8f6e-efce844f698f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:13.819984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244dfcb-490a-4159-8f6e-efce844f698f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:13.821419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 471/945] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:13.826974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b6b51-82b4-4292-909d-c91ac588d77c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:13.827267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b6b51-82b4-4292-909d-c91ac588d77c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:13.827545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b6b51-82b4-4292-909d-c91ac588d77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:13.846344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a18b6b51-82b4-4292-909d-c91ac588d77c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:13.847536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a18b6b51-82b4-4292-909d-c91ac588d77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:13.930802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 475/946] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:14.192455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f18a-a482-48ca-b9bc-cd3ee09234cf None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.192641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f18a-a482-48ca-b9bc-cd3ee09234cf None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.192923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f18a-a482-48ca-b9bc-cd3ee09234cf None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.194493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 472/947] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:14.199677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.199980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.200804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.207532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7f18a-a482-48ca-b9bc-cd3ee09234cf None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:14.221202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:14.222712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:14.238237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:14.313766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 476/948] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:14.326595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7f18a-a482-48ca-b9bc-cd3ee09234cf None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:14.361634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:14.362468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.362928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.363349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.364219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:14.365767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:14.368523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING keystone.server.flask.common [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 19:09:14.383477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:14.402752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.sql.core [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 19:09:14.402752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 19:09:14.402752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [parameters: {'id': 'fbfa67a32548454590ef99a9b5cdac32', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 19:09:14.402752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81559) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbfa67a32548454590ef99a9b5cdac32.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbfa67a32548454590ef99a9b5cdac32. Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:09:14.410884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result.read() Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 19:09:14.415598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application yield resource Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise self.value Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application session.commit() Sep 15 19:09:14.416591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.session.flush() Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application compat.raise_( Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 19:09:14.417507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 19:09:14.418421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application result.read() Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:09:14.420714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [parameters: {'id': 'fbfa67a32548454590ef99a9b5cdac32', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.422868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:14.424824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbfa67a32548454590ef99a9b5cdac32. Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.428675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 473/949] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Sep 15 19:09:14 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:14.429982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:14.538229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad572cb4-5217-41c5-ad85-f6b1c9d5f66e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:14.573467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:14.574150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.574484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.574766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.581565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:14.583009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:14.589105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:14.595423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:14.596360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:14.597281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 477/950] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:14.603193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:14.615261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:14.699349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a795f-4caa-4038-9cc9-d8684710ce43 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:14.735973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:14.736666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.737075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.737259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.737790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:14.739399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:14.743435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e122e-e467-497c-8ad4-5c6a0de8578d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:14.748650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:14.749794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:14.751165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:14.755257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 474/951] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:14.755688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:14.790457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:14.791860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.792295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.792716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.793551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:14.794547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:14.797166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:14.823926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 478/952] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:14.831569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:14.874980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c994dc-1363-46b3-9491-f330e94fe111 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:14.912795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:14.913422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:14.913704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:14.913979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:14.914624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:14.914927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:14.916425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:14.926210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 475/953] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:14.932070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:15.014375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:15.051463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaeb680-0a7f-4b16-b1c4-68957ee89fb5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:15.087930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:15.088547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.088829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.089113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.089742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:15.090051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:15.091462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cad7c-881f-4677-b35d-e4e1698d78f3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:15.097131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 479/954] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:15.103724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:15.217486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7019e-e191-4cce-9bc1-b635ffd23ea5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:15.245540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:15.253792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:15.254410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.254689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.254959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.255673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:15.257004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072d2af-723e-4c11-aacd-bd05b4bed81d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:15.308353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5d33-e637-414d-8997-fe32f7d1fea5 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.308543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5d33-e637-414d-8997-fe32f7d1fea5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.308821 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5d33-e637-414d-8997-fe32f7d1fea5 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.310468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 480/955] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:15.315366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5dbe9-13d2-4938-aa8f-6051973bd4d2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.315779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5dbe9-13d2-4938-aa8f-6051973bd4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.319853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5dbe9-13d2-4938-aa8f-6051973bd4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.335271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1ce5dbe9-13d2-4938-aa8f-6051973bd4d2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:15.336497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1ce5dbe9-13d2-4938-aa8f-6051973bd4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:15.351752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 476/956] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:15.430147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 481/957] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:15.595514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87b73e-2039-4ac8-913e-b3dc3bb87e9b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.595832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87b73e-2039-4ac8-913e-b3dc3bb87e9b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.596108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87b73e-2039-4ac8-913e-b3dc3bb87e9b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.597601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 477/958] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:15.603105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.603638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.604170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.623202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:15.624744 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:15.680300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:15.695357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 482/959] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:15.722076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87b73e-2039-4ac8-913e-b3dc3bb87e9b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:15.831205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87b73e-2039-4ac8-913e-b3dc3bb87e9b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:15.869044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:15.869675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:15.869910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:15.870133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:15.870725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:15.872657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:15.881805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:15.882924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:15.883809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 478/960] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:15.889185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:15.997097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd502095-b814-418b-b0d0-a557a8298b43 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.008680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.038757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.039474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.039793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.040070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.040702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:16.041007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.043081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.051804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 483/961] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:16.055312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.077735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:16.128388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:16.138403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc9f-5811-43de-9433-76aba6264583 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.182283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.183064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.183197 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.183489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.189568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145417-89a4-42aa-a211-5612fe1af95f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.201180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:16.209538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:16.210522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:16.211385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:16.212875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 479/962] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:16.220138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.237827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.238443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.238721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.238995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.239896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:16.240205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.242062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.251747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 484/963] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:16.345706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40aef-bba0-4ad5-ae8d-8f6db71924ea None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.356213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:16.383488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.384044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.384347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.384612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.385288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:16.385597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.386973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.398869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 480/964] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:16.404875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.515062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcca80-77d2-410c-bbd5-6d601b22021d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.533251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:16.551365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.552074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.558063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.558063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.563169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.564651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:16.580055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:16.581052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:16.581957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:16.581957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:16.581957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:16.581957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 485/965] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:16.587216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.692717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8861-6ada-4757-9b3f-eaa2f41d15d1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.729320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.730346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.730838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.731365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.732245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:16.732781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.734439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.751068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 481/966] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:16.759871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.862706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ed5a-562c-4d17-b191-2ad42730bc8e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:16.898602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:16.899574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:16.900089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:16.900660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:16.910319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:16.912144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:16.924283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:16.925219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 19:09:16.926257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:16.929390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 486/967] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:16.935827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:16.968624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.050341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c4bb-5834-485d-abcd-2edca0de1d5f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:17.088754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:17.089368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.089655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.089930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.091181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:17.091574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:17.093017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362aa5c-2736-4b00-9f7c-bbece2c80c1a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:17.115993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 482/968] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 15 19:09:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:17.117540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.147280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.165119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899d06-7b71-407a-abb2-31c1b5516198 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.165452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899d06-7b71-407a-abb2-31c1b5516198 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.165687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899d06-7b71-407a-abb2-31c1b5516198 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.167165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 483/969] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:17.173050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.173590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.174092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.196274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:17.198011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:17.242317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d88ca4-e288-4c06-b037-c73e7f20157c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:17.287713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 484/970] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:17.297793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:17.299760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.299760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.299760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.371049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:17.372793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:17.401248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.479120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 487/971] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:17.486768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.504556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.563516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.603379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b06c2c-9a4e-45e8-9e2e-e8a488442fed None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:17.641216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:17.642254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.642806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.643279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.654995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:17.657067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9ccfb081-5d9c-4ead-bf1f-de52fa0c957d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:17.673648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:17.674657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:17.675574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:17.675574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:17.675574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:17.676079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 485/972] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:17.677638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bf603-fb1c-45c9-879b-b3bd6f01af07 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.677638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bf603-fb1c-45c9-879b-b3bd6f01af07 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.677638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bf603-fb1c-45c9-879b-b3bd6f01af07 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.679217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 486/973] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:17.682155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bf603-fb1c-45c9-879b-b3bd6f01af07 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.691549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355b10-a210-4a50-a322-af5a98dbaf49 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:17.720833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.730844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:17.732113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.732397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.732671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.733395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:17.734747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:17.737326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING keystone.server.flask.common [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 19:09:17.757801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:17.785124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.sql.core [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 19:09:17.785124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 19:09:17.785124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [parameters: {'user_id': 'c9215800608f4094a9f6e449273b120d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 19:09:17.785124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81558) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 19:09:17.789054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bf603-fb1c-45c9-879b-b3bd6f01af07 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ab7ffaf2-898c-4044-8089-dbaf81d57422 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application result.read() Sep 15 19:09:17.801943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:09:17.802896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application yield resource Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application next(self.gen) Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise self.value Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application session.commit() Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 19:09:17.804424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self.session.flush() Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application compat.raise_( Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 19:09:17.805331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 19:09:17.806219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application conn.query(q) Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application result.read() Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 19:09:17.807088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c9215800608f4094a9f6e449273b120d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:17.808738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 19:09:17.809769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 488/974] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 15 19:09:17 2022] POST /identity/v3/users => generated 160 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.811021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.826372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:17.827940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:17.835094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:17.835769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:17.836057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:17.836384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:17.837030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:17.837336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:17.838690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:17.851378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 487/975] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:17.852225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.902206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 489/976] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:17.903606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:17.972753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8d46-d51b-4ff5-9bba-4efa803bad65 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.009375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.017990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.019190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.022568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.022568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.031638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.033775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.044150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfdac7-ab45-48bc-85a9-adb822e60b20 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:18.048822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:18.049753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 19:09:18.050678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:18.053769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 488/977] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:09:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:18.060854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.068521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.085020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.085963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.086477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.087054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.095320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.097360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:18.109307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:18.110259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:18.111174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:18.113621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 490/978] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:18.123704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.177098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6793a-caf4-45f0-bf16-fc60af4ca79a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.214427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.215035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.215320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.216164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.216881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.217188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.218601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.236036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8ee6-0783-4ffe-bce7-e4151ee61dd1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.263795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 489/979] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:18.263795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.283495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.284413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.285145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.285431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.286144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.286454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.288039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.311597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 491/980] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:18.317733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.325508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.384779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.386001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b57c-344f-4cf6-903a-4468e3900b34 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.431215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.432174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.432538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.432886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.433707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.435830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:18.446862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:18.448000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:18.448851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:18.454182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 490/981] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:18.455179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.473280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479503-6c68-4f92-8264-dd2606041600 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.518767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.519313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.519578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.519883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.520604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] RBAC: Adding query filter params (role.id=1dcd4d4337894b4ca66eaf9110717d66, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.520914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.522383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0526-3600-40e1-9c44-8905211538b2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.529333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 492/982] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/role_assignments?user.id=a2a29c5a3e2a43bb94c0bd21b33c9ff8&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1dcd4d4337894b4ca66eaf9110717d66 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:18.552920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8408d-5cfe-4aa4-92c3-3f9c3b05c306 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.553197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8408d-5cfe-4aa4-92c3-3f9c3b05c306 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.553468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8408d-5cfe-4aa4-92c3-3f9c3b05c306 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.555184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 493/983] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:18.561449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.561772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.561995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.583876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5177-dbcf-448e-84c4-e56e470dd340 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.584117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:18.585174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:18.607193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.633754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.634377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.634660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.634936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.635590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.635942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.637401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.645323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 491/984] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:18.651341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.688525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 494/985] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:18.764810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214287-c895-4323-84fe-bccfaba631a5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.799515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.800090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.800376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.800807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.801443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.801749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.803112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.808335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 492/986] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:18.814140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.849431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:18.925251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644416e-f61f-42a1-a3e7-8f430a6ada0f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:18.954335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:18.958483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:18.959098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:18.959381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:18.959687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:18.960381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] RBAC: Adding query filter params (interface=admin, service_id=6e3bdcd93224416db9da05523565e5e4, region_id=RegionOne) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:18.960692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:18.962087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:18.967499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 495/987] 10.222.0.18 () {56 vars in 1249 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/endpoints?service_id=6e3bdcd93224416db9da05523565e5e4&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:18.973142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:19.001087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:19.062717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e96f-ceaf-454c-ac14-cfb1b238b97e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:19.074383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:19.079520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544465e1-ce04-45ec-a3a7-e3944b78b359 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:19.097884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:19.098494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:19.098782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:19.099060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:19.099745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:19.101110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25fae7-f480-4969-be4d-69cec9732d33 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:19.114196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:19.114810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:19.115100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:19.115379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:19.116027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:19.117389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c6e80-e066-43a3-b2ae-c4e0cfc89be2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:19.124107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 496/988] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:18 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:09:19.171691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 493/989] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:18 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:20.727944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235db64-8cfb-4abf-b0cf-6f5ca2a9cea5 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:20.728525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235db64-8cfb-4abf-b0cf-6f5ca2a9cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:20.728525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235db64-8cfb-4abf-b0cf-6f5ca2a9cea5 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:20.729961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 497/990] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:20.735615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:20.736025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:20.736208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:20.754268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:20.755444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:20.831742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 494/991] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:21.096459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235db64-8cfb-4abf-b0cf-6f5ca2a9cea5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.204520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235db64-8cfb-4abf-b0cf-6f5ca2a9cea5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.239370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.240047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.240326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.240595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] PATH_INFO: `/v3/services/image` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.241166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.242508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-994c7612-b201-49ef-ab88-3eb9c18f448f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:21.251264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:21.252522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:21.253516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.255730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 498/992] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:21.258580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.290659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:21.295343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773c57-c2fc-43e9-b03e-98137e514476 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.295343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773c57-c2fc-43e9-b03e-98137e514476 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.295673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773c57-c2fc-43e9-b03e-98137e514476 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.297195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 499/993] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:21.302707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321ba0-eda1-4608-99ff-33463fd9f515 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.302991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321ba0-eda1-4608-99ff-33463fd9f515 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.303267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321ba0-eda1-4608-99ff-33463fd9f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.321321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4e321ba0-eda1-4608-99ff-33463fd9f515 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:21.322504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4e321ba0-eda1-4608-99ff-33463fd9f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:21.345021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:21.375683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f7313-3615-42ae-9889-44cfd06cb9c1 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.396727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 500/994] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:21.397988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc592d-9336-411e-8d17-c2cae2146b87 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.398281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc592d-9336-411e-8d17-c2cae2146b87 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.398560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc592d-9336-411e-8d17-c2cae2146b87 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.403675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 501/995] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:21.404769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.405056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.405332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.415238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.415871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.416156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.416432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.417059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] RBAC: Adding query filter params (name=image) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:21.417367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.418720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:21.423626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 495/996] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:21.425067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:21.426259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:21.429111 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.499717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 502/997] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:21.524651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:21.535227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78e13b-a361-4939-b954-bd0c94537f12 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.569868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.570463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.570743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.571016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.571635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] RBAC: Adding query filter params (type=image) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:21.571962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.573298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:21.579792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 496/998] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:21.671299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.771229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:21.782843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.788312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9c9f0-9155-4752-bad5-c8cbbc51e1e3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.824334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.825501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.825781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.826052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.826642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.828047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:21.837051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:21.838460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:21.839313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 503/999] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:21.844538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.900485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fc79-6650-4864-8e17-e4e5fef7efce None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.943085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.943902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.944373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.944499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.945096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.946830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:21.952896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181f7f5-522e-44d1-ac86-96ea84cede1c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:21.957787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:21.958827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:21.959681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 497/1000] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:21.964655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:21.987583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:21.991612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:21.992247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:21.992529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:21.992799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:21.993422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:21.993726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:21.995128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.005284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 504/1001] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:22.006789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:22.024234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:22.077792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9f003-b02a-4614-9bfb-d224ca7ad47c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.112893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.113505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.113791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.114068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.114729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:22.115044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.116449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.119318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5107a7f-4dfd-4efb-b298-9e4ec5f752fc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.121225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 498/1002] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:22.127276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:22.159273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.159901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.160182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.160457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.161082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:22.161384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.162741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.167474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 505/1003] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 15 19:09:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:22.173543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:22.216249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:22.236095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a77c3-55ae-49da-b1cc-71af0bebe1ad None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.256601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:22.276962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.277566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.277852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.278157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.278787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:22.279094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.279359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96187f-a37a-4cf7-8f0b-02e08ef268b8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.280473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.285789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 499/1004] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:22.291556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:22.318141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.318747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.319025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.319299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.320017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.321369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.394517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985131d-4e63-4445-ac19-f6ffdcca944a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.410012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 506/1005] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:22.449332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.449976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.450260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.450540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.451286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] RBAC: Adding query filter params (interface=public, service_id=6bc9915d00ee4df28cd4542eb52a7d16, region_id=RegionOne) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:22.451603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.453278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee1965-97a6-4a6c-8a71-f76ea4802be1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.463965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 500/1006] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:09:22 2022] GET /identity/v3/endpoints?service_id=6bc9915d00ee4df28cd4542eb52a7d16&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:22.466962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:22.579414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb258985-3f8f-4f04-94a3-4b9805a67f17 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:22.615015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:22.615615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:22.615923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:22.616197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:22.616793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:22.618145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b0829-8307-49f8-8b8d-642a3f9e4e08 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:22.624504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 507/1007] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:22 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:09:23.749868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04defb-0081-4721-9b05-df32fd02931f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:23.750421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04defb-0081-4721-9b05-df32fd02931f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:23.750421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04defb-0081-4721-9b05-df32fd02931f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:23.751829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 501/1008] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:23.756995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581d34c-1126-417e-adc0-a2b14267b734 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:23.757439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581d34c-1126-417e-adc0-a2b14267b734 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:23.757589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581d34c-1126-417e-adc0-a2b14267b734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:23.764152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:23.776969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2581d34c-1126-417e-adc0-a2b14267b734 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:23.778681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2581d34c-1126-417e-adc0-a2b14267b734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:23.847689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 508/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:24.115675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04defb-0081-4721-9b05-df32fd02931f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:24.125780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:24.219681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04defb-0081-4721-9b05-df32fd02931f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:24.256181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:24.257079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:24.257543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:24.257994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:24.258886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:24.260539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5da87-294d-45a7-aab2-992a90cbef47 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:24.365119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 502/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:24 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:24.618839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1def6-947f-4fb6-8097-f2025efb5591 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:24.619227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1def6-947f-4fb6-8097-f2025efb5591 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:24.619406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1def6-947f-4fb6-8097-f2025efb5591 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:24.620810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 509/1011] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:24.628562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2509ac-57bc-47b1-afa9-06abc8be3cc4 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:24.628837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2509ac-57bc-47b1-afa9-06abc8be3cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:24.629112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2509ac-57bc-47b1-afa9-06abc8be3cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:24.647228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-da2509ac-57bc-47b1-afa9-06abc8be3cc4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:24.648457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-da2509ac-57bc-47b1-afa9-06abc8be3cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:24.723075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:24.737717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 503/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:24 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:24.795327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6b776-f83f-4f24-9eba-279d7fe783d0 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:24.795877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6b776-f83f-4f24-9eba-279d7fe783d0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:24.795956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6b776-f83f-4f24-9eba-279d7fe783d0 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:24.797408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 510/1013] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:24.802444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:24.802830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:24.803004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:24.821245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:24.822439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:24.892220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 504/1014] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:24 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:25.005371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6b776-f83f-4f24-9eba-279d7fe783d0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.109180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6b776-f83f-4f24-9eba-279d7fe783d0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.139157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:25.144779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.145350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.145707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.145970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.147147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.148663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.150315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 511/1015] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:25.156761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.164806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.265160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bbd10-cf37-47d5-a399-56a42f8a9ecc None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.272011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a1641-82dd-4bd0-8873-7f959765a5e9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.290311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:25.302752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.303361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.303642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.303942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.304602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.306027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.310436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.311229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.311719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.312151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.312911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.314408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:25.324273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:25.326071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:25.326941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 512/1016] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:25.332417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.392998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 505/1017] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:25 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:25.404119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:25.445174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e19810-d474-4261-bb53-33f261fce907 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.482251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.482875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.483158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.483435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.484104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] RBAC: Adding query filter params (name=compute) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:25.484410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.485805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.492377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 513/1018] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:25.498468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.603293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add679bc-a074-4e36-b538-cd4a8b3c04a1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.639226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.639862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.640146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.640425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.641051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:25.641498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.642789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bd928-f80d-4148-a3f5-28724ade5faa None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.647964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 506/1019] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 15 19:09:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:25.653888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:25.755969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e9b59-dbbd-47ee-a819-a60d39b3dbde None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:25.783228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:25.791571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:25.792207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:25.792490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:25.792763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:25.793463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:25.794826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb500e0-c54b-4394-a883-73f324c40516 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:25.875671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 514/1020] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:26.558324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5e3a-b2a0-425e-8b67-e9d2f986a0bb None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:26.558852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5e3a-b2a0-425e-8b67-e9d2f986a0bb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:26.558932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5e3a-b2a0-425e-8b67-e9d2f986a0bb None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:26.560979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 507/1021] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:26.566216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:26.566506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:26.566784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:26.584910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:26.586080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:26.660751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 515/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:26.927838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea5e3a-b2a0-425e-8b67-e9d2f986a0bb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:26.970615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:27.031150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea5e3a-b2a0-425e-8b67-e9d2f986a0bb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:27.068291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:27.068923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.069208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.069481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] PATH_INFO: `/v3/services/image` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.070062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:27.071407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:27.080601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:27.081583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:27.082436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 508/1023] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 15 19:09:26 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:27.089275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:27.191105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ef6e1-2e3a-4144-b6fb-b14ca109cbe2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:27.202628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:27.226611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:27.227231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.227515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.227826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.228455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] RBAC: Adding query filter params (name=image) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:27.228759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:27.230121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:27.235409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 516/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:27.240908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:27.344592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f1d06-5bf7-468d-8fc9-119892c40424 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:27.346144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:27.379830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:27.380438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.380724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.381001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.381624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] RBAC: Adding query filter params (type=image) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:27.381928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:27.383280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:27.388795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 509/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:27.394434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:27.494011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9291675-d6e2-4842-b958-e94bd84a88be None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:27.533392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:27.534005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.534291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.534568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.535271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] RBAC: Adding query filter params (interface=public, service_id=896e8133049a4ce78a206b740af7a9da, region_id=RegionOne) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:27.535586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:27.537032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5ea5-eac7-46aa-bd04-b89889023a4e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:27.542346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 517/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:09:27 2022] GET /identity/v3/endpoints?service_id=896e8133049a4ce78a206b740af7a9da&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:27.547912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:27.619150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1bcf-7700-4d93-9bd7-98460777121a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.619437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1bcf-7700-4d93-9bd7-98460777121a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.619748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1bcf-7700-4d93-9bd7-98460777121a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.621097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 518/1027] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:27.627548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c391-7294-41d5-aaea-f4d1b2acd7fa None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.627872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c391-7294-41d5-aaea-f4d1b2acd7fa None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.628150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c391-7294-41d5-aaea-f4d1b2acd7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.646700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2d10c391-7294-41d5-aaea-f4d1b2acd7fa None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:27.647893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2d10c391-7294-41d5-aaea-f4d1b2acd7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:27.650902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3f870-2ce2-43ed-a101-a78527ba8621 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:27.679799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:27.686484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:27.687098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:27.687381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:27.687668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:27.688301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:27.689688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:27.696546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 510/1028] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:09:27 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:09:27.722145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 519/1029] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:27.990611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:28.023419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:28.097069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fb7d4-075d-43a0-928e-5bd53e6901ea None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:28.131596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:28.132240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.132521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.132794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.133881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:28.135289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:28.137079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 511/1030] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:28.142144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba757ce-2b6b-4633-accc-f8d20df73726 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.142608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:28.142699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba757ce-2b6b-4633-accc-f8d20df73726 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.142699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba757ce-2b6b-4633-accc-f8d20df73726 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.144206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 520/1031] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:28.148835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.149121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.149395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.171011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:28.172207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:28.241864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 521/1032] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:28.248685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:28.253367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1ff9-1d44-4112-82bb-64ccb2ac3508 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:28.287827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:28.288432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.288714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.288988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.289696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:28.291107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:28.397357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 512/1033] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:09:28 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:28.511355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:28.617815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:28.621276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101eb007-cb0e-456e-a82d-41d7a570d8ac None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:28.657857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:28.658493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.658777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.659051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.670378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:28.675342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:28.676331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:28.677205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 522/1034] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:28.682572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:28.785626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297782e-c34c-4855-9a89-5000b32dd34f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:28.818398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:28.819001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.819281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.819613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:28.820304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:28.820654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:28.822004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:28.835210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 513/1035] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:28.836860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:28.947723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47160b17-f5fc-49ed-aeac-92f1a1b96354 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:28.986382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:28.987387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:28.987994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:28.988492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.000475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:29.002187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:29.017069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:29.018027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:29.018933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:29.018933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:29.018933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:29.025343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 523/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:29.026496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:29.045954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:29.129134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f053b1-411c-4c52-a49f-116214eb5d00 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:29.164292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:29.165233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.165731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.166215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.167046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:29.167507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:29.169578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:29.181064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 514/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:29.187509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:29.201296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:29.288748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca5717-a2c3-4fad-aab9-f4e2b9f3ad83 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:29.322494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:29.323052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.323332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.323602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.331805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:29.333300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:29.345973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:29.347091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:29.348081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:29.351901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 524/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:09:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:29.353239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:29.430694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:29.453761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c2d2-b8cd-4108-a98b-5c1d3ad13df3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:29.488677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:29.489283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.489566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.489841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.490561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:29.490869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:29.492317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:29.514469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 515/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:09:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:29.516244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:29.586161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:29.617745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0b08-e6db-447c-879d-b1b1317243c3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:29.651831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:29.652433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.653235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.653515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.725000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:29.726451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc3885-3802-404c-9932-108b4da1cd0e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:29.802603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:29.834794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 525/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 19:09:29.840707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:29.888597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b850-c256-4c65-84a6-19d4a6568c1c None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.888888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b850-c256-4c65-84a6-19d4a6568c1c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.889172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b850-c256-4c65-84a6-19d4a6568c1c None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.890513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 526/1041] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:29.895340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.895638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.895951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.917245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:29.919072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:29.953050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5444b90-cfff-4aac-a1e1-ae371e3f3493 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:29.987601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:29.988270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:29.988554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:29.988829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:29.989998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 527/1042] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:29.992643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.002333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.003826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:30.018930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:30.019933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:30.020853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:30.020853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:30.020853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:30.021582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 516/1043] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:30.026013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.137102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.140233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fbd85-bb6c-4755-b19a-86a44a5dac2c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.175841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.176447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.176729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.177002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.177640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:30.177950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.179385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.195607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 528/1044] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:30.197799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.268738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.301779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d00e-78bf-4080-a14b-1951eccbee57 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.339150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.340377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.340664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.340945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.348184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.349756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:30.365034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:30.365981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:30.366893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 517/1045] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:30.372567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.374786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79bca0-c190-4add-b243-7ae10f9698f5 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.411440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.412021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.412304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.412639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] PATH_INFO: `/v3/services/image` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.413227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.415121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.421659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:30.427066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:30.437883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:30.438738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 529/1046] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:30.439559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.463363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.481787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833434-485b-4db8-9b47-f827600c7037 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.524930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.525553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.525826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.526103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.526925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:30.527135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.529058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.547295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 518/1047] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:30.547467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701961-a07b-4eb2-9629-70452008d476 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.548407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.583563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.584203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.584484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.584764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.585392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:30.585700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.587067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.596607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 530/1048] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:30.597601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.656960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8061f-3d48-4648-b409-59cec1f2a76a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.664821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.686326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:30.691275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.691907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.696025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.696025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.696025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] RBAC: Adding query filter params (role.id=1dcd4d4337894b4ca66eaf9110717d66, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:30.696025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.696025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314eec-d0d4-4c14-9d71-c3a84c053b15 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.698240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6f22-1436-43c1-9de6-f3f319021df6 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 519/1049] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1dcd4d4337894b4ca66eaf9110717d66 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7956a-1020-4c33-a040-41d2dcc725ef None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7956a-1020-4c33-a040-41d2dcc725ef None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7956a-1020-4c33-a040-41d2dcc725ef None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 520/1050] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:30.715917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.716528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.716528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.730094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:30.731379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:30.738068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.738730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.739014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.739287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.740672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] RBAC: Adding query filter params (type=image) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:30.740985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.742889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:30.749411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 531/1051] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:30.756109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:30.806222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 521/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:30.864834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cdb2-c568-4c6e-a234-61c8509d501a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:30.901613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:30.902238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:30.902510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:30.903359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:30.904067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:30.905437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:31.013815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 532/1053] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:09:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:09:31.080957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:31.194561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:31.198144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679354-8cfd-4b0f-a0bb-d3a98b1a4764 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:31.240280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:31.240912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:31.241199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:31.241471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:31.251776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:31.256568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:31.257486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:31.258362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 522/1054] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:31.265792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:31.277489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:31.370132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96184b-050b-40d8-98c5-899162efe208 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:31.404294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:31.404901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:31.405183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:31.405457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:31.406383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:31.406688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:31.408100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:31.424982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 533/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:31.425175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:31.526850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738780-9f74-48d7-b780-76a59be3a050 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:31.561994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:31.562622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:31.562904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:31.563180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] PATH_INFO: `/v3/users/250e731ccc03447ebde85ac30585e903` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:31.578485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] RBAC: Authorizing `identity:get_user(user_id=250e731ccc03447ebde85ac30585e903)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:31.579995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:31.582210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 523/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:31 2022] GET /identity/v3/users/250e731ccc03447ebde85ac30585e903 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:31.588067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:31.659190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:31.689120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725f106-8ca2-4025-8679-445c8114229f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:31.725610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:31.726227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:31.726508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:31.726784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:31.801381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:31.802754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:31.885296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:31.924270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 534/1057] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:31.934289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:31.951214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:32.053073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6479-5ac1-44ae-9727-27f1d9cf6bbc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:32.088862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:32.090336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:32.090825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:32.091322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:32.092667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:32.094285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b491bd-46f4-4a57-bcff-366583106456 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:32.096175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 524/1058] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:31 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:32.102556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:32.205703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1e12-e106-4f54-906d-0df51df2c448 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:32.239159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:32.239791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:32.240076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:32.240348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:32.241105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c, user.id=250e731ccc03447ebde85ac30585e903) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:32.241420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:32.242837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8d01f-8b6a-43b0-802e-df736f673db7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:32.251947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 535/1059] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:32 2022] GET /identity/v3/role_assignments?user.id=250e731ccc03447ebde85ac30585e903&scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:32.862217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89226b2-e8d6-4f47-9e8a-9406985673d9 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:32.862644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89226b2-e8d6-4f47-9e8a-9406985673d9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:32.862821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89226b2-e8d6-4f47-9e8a-9406985673d9 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:32.864370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 525/1060] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:32.869352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a61f5-e39a-4ec8-83f5-5aba320d3e94 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:32.869801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a61f5-e39a-4ec8-83f5-5aba320d3e94 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:32.869974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a61f5-e39a-4ec8-83f5-5aba320d3e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:32.889051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-573a61f5-e39a-4ec8-83f5-5aba320d3e94 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:32.890233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-573a61f5-e39a-4ec8-83f5-5aba320d3e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:32.919044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:32.967692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 536/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:33.217383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9df58-afb0-470a-abe1-f3a42636b21b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.217885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9df58-afb0-470a-abe1-f3a42636b21b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.217885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9df58-afb0-470a-abe1-f3a42636b21b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.219180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 526/1062] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:33.224877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.225337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.225623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.237195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9df58-afb0-470a-abe1-f3a42636b21b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.244442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:33.245634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:33.315284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 537/1063] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:33.327338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:33.346288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9df58-afb0-470a-abe1-f3a42636b21b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:33.379662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:33.380294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.380580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.380856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.391681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:33.396433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:33.397364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:33.400626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 527/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:33.406211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.506638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:33.514171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d326-41e6-46b5-8029-3b4d3e31e069 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:33.549603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:33.550231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.550521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.550794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.551480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:33.551875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:33.553241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:33.565886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 538/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:33.567034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.583054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.678344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1a58-0a82-4f0a-a63c-dbfc151b5522 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:33.692046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d2d25-02e6-4db5-b95c-ace721ae21cc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:33.712941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:33.713568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.713847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.714145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.725234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:33.728357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:33.728504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:33.742359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:33.742603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:33.747217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:33.748155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:33.748155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:33.748155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:33.748345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.748345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.748345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.748535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 528/1066] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:33.749836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.779998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:33.826245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:33.827985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:33.855701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d184e06-c1a4-44ef-a4ed-2cd570172f33 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:33.892786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:33.893554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:33.893837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:33.894575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:33.895223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:33.895697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:33.897062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:33.914729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 529/1067] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:33.920107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.932457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 539/1068] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:33.937935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:33.983987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.000719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.031591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2951f9-fd0a-428f-897a-2c42ab85f1e8 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.050947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d634c12-500b-4197-b0ee-f5f7ac28c0af None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.066937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.067828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.068480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.068772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.077984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.079411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.088246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.088868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.089146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.089418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:34.090535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:34.091447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:34.092456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 530/1069] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:34.097333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.103213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.104705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:34.120520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:34.121696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:34.124635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:34.124635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:34.124635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:34.124635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 540/1070] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:34.127702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.208905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabac0-a979-4d27-8853-569391cea52e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.224653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.233475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c043-06e6-4c10-ac60-59e2f963261f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.243415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.246792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.247438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.247684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.247965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.248689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:34.248995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.250479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.271229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 531/1071] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:34.272284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.273086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.273515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.273862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.274601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:34.274971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.276492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e3f83-feb7-4613-827d-f2a6fd67fb8c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.276982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.292887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 541/1072] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:34.391432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597a8f9-a017-4800-b8bf-e530f809ecac None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.393625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343299-93d1-4d33-a2fe-f21f99eb5d15 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.393920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343299-93d1-4d33-a2fe-f21f99eb5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.395093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343299-93d1-4d33-a2fe-f21f99eb5d15 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.396550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 542/1073] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:34.403279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.403619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.403869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.427823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:34.429648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:34.430700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.431350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.431633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.431942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.444273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.446283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:34.463396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:34.464646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:34.466161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:34.466161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:34.466161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:34.467703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 532/1074] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:34.473438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.503010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.511058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 543/1075] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:34.585686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893a7-3887-4a59-9699-767299d382b2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.623725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.624275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.624555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.624824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.625657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:34.625849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.627190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.639125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 533/1076] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:34.644943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.738082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.741923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e85df-d0f6-4a3a-9d1a-d43edd259d5d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.775698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.776415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.776693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.777036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.777305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.784810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.786276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:34.798292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:34.804052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 544/1077] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:34.805250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.867599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:34.883200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cab8-3e6d-4585-a4e5-e227acbb96a9 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.903630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc816ad-71a5-4e14-bcf8-7c9111dd727d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:34.920278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.920915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.921203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.921482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.931555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:34.936454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:34.937387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:34.938790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 534/1078] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:34.940378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:34.941046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:34.941399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:34.941732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:34.942484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:34.942851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:34.943391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.944300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:34.955727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 545/1079] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:34.961441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:34.980252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:35.040949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a472a-c560-48dc-ba44-dc74d0b4493f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:35.059876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319a69-d66e-4ca4-a526-9b5d56110139 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:35.074120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:35.074728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:35.075013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:35.075294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:35.075979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:35.076287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:35.077638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:35.085505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 535/1080] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:35.091711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:35.093526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:35.094271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:35.094464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:35.094736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/1dcd4d4337894b4ca66eaf9110717d66` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:35.096914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=c9b27eb952a04c46bb36653fb79abe3e, role_id=1dcd4d4337894b4ca66eaf9110717d66)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:35.098348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:35.100452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:35.161665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 546/1081] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:09:34 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/c9b27eb952a04c46bb36653fb79abe3e/roles/1dcd4d4337894b4ca66eaf9110717d66 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:09:35.192865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8d3e9-f616-4a80-8865-1141e94c041c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:35.230174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:35.231465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:35.231677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:35.231956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] PATH_INFO: `/v3/users/250e731ccc03447ebde85ac30585e903` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:35.233161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] RBAC: Authorizing `identity:get_user(user_id=250e731ccc03447ebde85ac30585e903)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:35.234598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:35.236276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 536/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:35 2022] GET /identity/v3/users/250e731ccc03447ebde85ac30585e903 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:35.242284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:35.333452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:35.354992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094695c7-c0d4-49d3-9924-6d6a2bbcc69a None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:35.389681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:35.390332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:35.390612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:35.390886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:35.392024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:35.393455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb28bdd-9a81-49a3-8463-829f8ffc1ed2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:35.395224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 547/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:35 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:35.401228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:35.475830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:35.499837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f53aab-3f42-483f-95d5-f64722e37db0 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:35.534322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:35.534978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:35.535262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:35.535541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c/users/250e731ccc03447ebde85ac30585e903/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:35.537453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5186e9d514ce9bca6536139f8fb1c, user_id=250e731ccc03447ebde85ac30585e903, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:35.538874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b98b36-f1e1-4799-ad47-86473f2e38a9 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:35.602047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 537/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:09:35 2022] PUT /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c/users/250e731ccc03447ebde85ac30585e903/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:09:37.306915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df083490-5cf4-4563-91d2-3c77ed08a9d8 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.307470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df083490-5cf4-4563-91d2-3c77ed08a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.307470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df083490-5cf4-4563-91d2-3c77ed08a9d8 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.309542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 548/1085] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:37.314911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2575-abb6-44ae-952f-b1e310568dde None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.315315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2575-abb6-44ae-952f-b1e310568dde None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.315471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2575-abb6-44ae-952f-b1e310568dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.334486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-053c2575-abb6-44ae-952f-b1e310568dde None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:37.335691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-053c2575-abb6-44ae-952f-b1e310568dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:37.400935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 538/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:37.674736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df083490-5cf4-4563-91d2-3c77ed08a9d8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:37.726640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7353bc-332b-4494-9610-7bdf3e4f7195 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.728324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7353bc-332b-4494-9610-7bdf3e4f7195 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.728324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7353bc-332b-4494-9610-7bdf3e4f7195 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.728609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 539/1087] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:09:37.734113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.734434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.734705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.752991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:37.754171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:37.779582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df083490-5cf4-4563-91d2-3c77ed08a9d8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:37.812463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:37.814274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:37.814900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.815175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.815445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.820620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 540/1088] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:37.826182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:37.830956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:37.836541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:37.837475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 549/1089] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 15 19:09:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:37.838181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:37.936971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b167b-20d7-4f5d-8b6a-6c716f710a6d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:37.970083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:37.970734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:37.971001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:37.971275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:37.971966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:37.972271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:37.973631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:37.981963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 541/1090] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:09:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:37.987767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:37.996946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.088032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f9ec-7c3c-4758-8134-c3335f30c853 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.088823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.122562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.123190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.123475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.123781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.134054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.135608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.144252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:38.150082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:38.151217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:38.152692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:38.152692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:38.152692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:38.152692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 550/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:38.157111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.187572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67658966-4960-43df-8d26-7f8c2b60abfe None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.213219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.221029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.221659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.221940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.222214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.232693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:09:38.237946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:38.238871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:09:38.239757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:38.269559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 542/1092] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:38.269559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.270303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9ceef-0ff5-42e2-9565-d1b95e6fb475 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.290210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.290818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.291104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.291379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.292193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:38.292504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.294410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.306158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 551/1093] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:38.313722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.359484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38362f97-d143-41f8-a4b1-ae5964e3bf1c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.377647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.395301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.395963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.396251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.396840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.397509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:38.397817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.399180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.411888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 543/1094] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:38.414524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.418255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2f58d-9d1b-4a4c-8eea-c4d49fae4ab1 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.435682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.460442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.460442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.460442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.460442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.460933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.462223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:09:38.474120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:38.475099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:38.476111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 552/1095] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:38.481402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.517462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd2594-d170-4fac-8fdf-88d48ea77e86 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.550369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.551013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.551293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.551561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] PATH_INFO: `/v3/users/250e731ccc03447ebde85ac30585e903` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.552739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] RBAC: Authorizing `identity:get_user(user_id=250e731ccc03447ebde85ac30585e903)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.554180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.555758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 544/1096] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/users/250e731ccc03447ebde85ac30585e903 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:38.561900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.582926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb7897-f267-4aa9-9c4e-cc10cf289583 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.606667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.616219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.616818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.617096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.617368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.618092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:38.618398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.619836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.638654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 553/1097] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:38.644339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.660567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e18606-7696-451c-90c9-2525964749b6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.678579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.694568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.695149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.695434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.695741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.742107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614dcb8-8894-471c-adf1-27e92abd93c7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:38.761073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.762522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.775242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:38.775874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:38.776154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:38.776434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:38.807886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.842749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:38.844324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:38.860596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 545/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 19:09:38.866521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.881670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:38.947417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 554/1099] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 19:09:38.953940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:38.973967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226060bd-885d-4448-b51b-a90717150ab5 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.007398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.007985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.008267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.008548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.009745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.011189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.013440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 546/1100] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:39.020338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:39.035722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:39.052457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e113b-ed46-46a3-a606-17d78ddf07a2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.085961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.086586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.086867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.087149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.089743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:39.098073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.100572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:09:39.114074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:39.115158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:09:39.116754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:09:39.116754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:09:39.116754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:09:39.116754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 555/1101] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:09:39.120880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:39.124050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb5124-d307-4cac-9205-2741a4392e7b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.158911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bd813-071b-4568-826a-5d6379042906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.159510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bd813-071b-4568-826a-5d6379042906 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.159824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bd813-071b-4568-826a-5d6379042906 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.160100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bd813-071b-4568-826a-5d6379042906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.160815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bd813-071b-4568-826a-5d6379042906 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c, user.id=250e731ccc03447ebde85ac30585e903) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:39.161126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bd813-071b-4568-826a-5d6379042906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.162551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bd813-071b-4568-826a-5d6379042906 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.168816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 547/1102] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:39 2022] GET /identity/v3/role_assignments?user.id=250e731ccc03447ebde85ac30585e903&scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:39.232097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f4257-0191-47e8-81a5-3acc4137517e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.268263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.268803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.269086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.269362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.270015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:39.270322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.271726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.285360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 556/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:09:39.291336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bd813-071b-4568-826a-5d6379042906 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:39.374538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:39.396610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bd813-071b-4568-826a-5d6379042906 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.430834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.431454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.431767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.432044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.438537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.439988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:09:39.450938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:09:39.452046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:09:39.453321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 548/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:09:39.457992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:39.506986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:39.556726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675badd-3d78-48a6-a323-ea7f2cb03fd9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.590337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.590937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.591219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.591490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.592197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:39.592512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.593924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04405f93-52f6-487a-8abc-a75b8cee531a None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.605129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 557/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:09:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:39.611023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:39.710195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d0eb-52a7-4bdf-aa61-ebaa025c0998 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:39.742802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:39.743405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:39.743716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:39.743995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:39.744827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] RBAC: Adding query filter params (role.id=1dcd4d4337894b4ca66eaf9110717d66, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=c9b27eb952a04c46bb36653fb79abe3e) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:39.745141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:39.747112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421a146-52cf-49a4-a435-e784b967b365 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:39.752838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 549/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:39 2022] GET /identity/v3/role_assignments?user.id=c9b27eb952a04c46bb36653fb79abe3e&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1dcd4d4337894b4ca66eaf9110717d66 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:41.354289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019c2c-f485-414e-944e-be2f7f3ee2da None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:41.354846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019c2c-f485-414e-944e-be2f7f3ee2da None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:41.354846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79019c2c-f485-414e-944e-be2f7f3ee2da None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:41.356762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 558/1107] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:41.361898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:41.362341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:41.362446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:41.368717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:41.381322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:41.382483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:41.446936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 550/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:41.716411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79019c2c-f485-414e-944e-be2f7f3ee2da None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:41.817022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79019c2c-f485-414e-944e-be2f7f3ee2da None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:41.825578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:41.849727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:41.850351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:41.850665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:41.850941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:41.852071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:41.853548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b1b07-fa2c-4925-b60f-d0e6fc1c2259 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:41.855190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 559/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:41.861503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:41.956477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fba033-5ee1-44d1-b10d-aa660aa6a9b0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:41.985157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:41.989949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:41.990553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:41.990836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:41.991108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:41.991782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:41.993194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d09e9-b949-43d0-9390-aefd6fb6cbc3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:42.059531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 551/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:09:41 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:44.218114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd142-337c-4191-ba21-342f0c73440c None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:44.218623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd142-337c-4191-ba21-342f0c73440c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:44.218623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd142-337c-4191-ba21-342f0c73440c None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:44.219993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 560/1111] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:44.225232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390870a-5443-4221-b719-2d561da307bf None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:44.225651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390870a-5443-4221-b719-2d561da307bf None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:44.225800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390870a-5443-4221-b719-2d561da307bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:44.243427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8390870a-5443-4221-b719-2d561da307bf None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:44.244711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8390870a-5443-4221-b719-2d561da307bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:44.309853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 552/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:44.578016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd142-337c-4191-ba21-342f0c73440c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:44.677389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd142-337c-4191-ba21-342f0c73440c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:44.709087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:44.709707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:44.709992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:44.710267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] PATH_INFO: `/v3/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:44.711639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] RBAC: Authorizing `identity:get_role(role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:44.713032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:44.714333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 561/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:44 2022] GET /identity/v3/roles/7647dfe2bb394802a35379f976d57e94 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:44.720600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390870a-5443-4221-b719-2d561da307bf None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:44.817652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390870a-5443-4221-b719-2d561da307bf None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:44.835161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:44.852014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:44.852635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:44.852921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:44.853200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] PATH_INFO: `/v3/users/0fcb37faf8f14a0e97e6572b6624ead3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:44.866987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] RBAC: Authorizing `identity:get_user(user_id=0fcb37faf8f14a0e97e6572b6624ead3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:44.868477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:44.870063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 553/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:44 2022] GET /identity/v3/users/0fcb37faf8f14a0e97e6572b6624ead3 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:44.875808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:44.922075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:44.975299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad474-b959-4d51-8ad3-e8c4874851dc None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:45.008296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:45.008889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:45.009169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:45.009441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:45.073664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:45.075607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:45.187600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:45.241891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 562/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:45.247931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:45.343358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308500-d88b-4494-83ee-a6551a3b9b5a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:45.375493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:45.376138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:45.376425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:45.376701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:45.377797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:45.379211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd7950-d849-4398-85a4-384bc2cf38fe None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:45.380863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 554/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:45 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:45.387447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:45.482506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d235c0-169d-44ec-ad77-d2e2bce26671 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:45.516743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:45.517333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:45.517613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:45.517885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:45.518595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c, user.id=0fcb37faf8f14a0e97e6572b6624ead3) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:45.518899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:45.520338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6787c-a7ad-4108-ac87-1ee01704b4a4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:45.525613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 563/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:45 2022] GET /identity/v3/role_assignments?user.id=0fcb37faf8f14a0e97e6572b6624ead3&scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:47.644346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07c87-f0af-49b0-942d-c22efc2d8c60 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:47.644878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07c87-f0af-49b0-942d-c22efc2d8c60 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:47.644878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07c87-f0af-49b0-942d-c22efc2d8c60 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:47.646223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 555/1118] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:47.651583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:47.651989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:47.652205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:47.670159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:47.671337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:47.673281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:47.736420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 564/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:09:48.007767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07c87-f0af-49b0-942d-c22efc2d8c60 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:48.053870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:48.106765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07c87-f0af-49b0-942d-c22efc2d8c60 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:48.139083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:48.139867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:48.140168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:48.140452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] PATH_INFO: `/v3/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:48.141842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] RBAC: Authorizing `identity:get_role(role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:48.143194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:48.144532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 556/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:48 2022] GET /identity/v3/roles/7647dfe2bb394802a35379f976d57e94 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:48.150290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:48.245149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ffc53-8094-47e8-aa87-70edb1ea32a5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:48.277566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:48.278175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:48.278453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:48.278723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] PATH_INFO: `/v3/users/0fcb37faf8f14a0e97e6572b6624ead3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:48.279857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] RBAC: Authorizing `identity:get_user(user_id=0fcb37faf8f14a0e97e6572b6624ead3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:48.281527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:48.283054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 565/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:48 2022] GET /identity/v3/users/0fcb37faf8f14a0e97e6572b6624ead3 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:48.288571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:48.383123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443288-73e4-4d7f-b847-99f60cb5b671 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:48.395123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:48.415329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:48.415969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:48.416250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:48.416521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:48.417626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:48.419782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be41bd-6afb-4609-9aa4-9460fecfd333 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:48.421380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 557/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:48 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:48.427317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:48.436085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:48.524402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc69b7-8190-4f43-b741-3325d5a8a71d None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:48.557439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:48.558087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:48.558398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:48.558678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c/users/0fcb37faf8f14a0e97e6572b6624ead3/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:48.560820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5186e9d514ce9bca6536139f8fb1c, user_id=0fcb37faf8f14a0e97e6572b6624ead3, role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:48.562346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b60eb-4403-4068-bf93-ac5fec8236c9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:48.624506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 566/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:09:48 2022] PUT /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c/users/0fcb37faf8f14a0e97e6572b6624ead3/roles/7647dfe2bb394802a35379f976d57e94 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:09:50.712317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694eb77-38a8-4100-b0a7-db8ca10e8c97 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:50.712851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694eb77-38a8-4100-b0a7-db8ca10e8c97 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:50.712851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694eb77-38a8-4100-b0a7-db8ca10e8c97 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:50.714195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 558/1124] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:50.719566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:50.720016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:50.720157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:50.737873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:50.739037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:50.796202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:50.804888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 567/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:51.077735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694eb77-38a8-4100-b0a7-db8ca10e8c97 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:51.174975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694eb77-38a8-4100-b0a7-db8ca10e8c97 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:51.206760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:51.207363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:51.207642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:51.207937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] PATH_INFO: `/v3/roles/7647dfe2bb394802a35379f976d57e94` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:51.209293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] RBAC: Authorizing `identity:get_role(role_id=7647dfe2bb394802a35379f976d57e94)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:51.210635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:51.211944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 559/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:51 2022] GET /identity/v3/roles/7647dfe2bb394802a35379f976d57e94 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:51.218274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:51.317911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c965e-e8cd-42ca-93b8-88002fe2c809 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:51.350913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:51.351477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:51.351785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:51.352060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] PATH_INFO: `/v3/users/0fcb37faf8f14a0e97e6572b6624ead3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:51.353165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] RBAC: Authorizing `identity:get_user(user_id=0fcb37faf8f14a0e97e6572b6624ead3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:51.354591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:51.356711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 568/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:09:51 2022] GET /identity/v3/users/0fcb37faf8f14a0e97e6572b6624ead3 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:51.362432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:51.410412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:51.458989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754cdd4-31e8-4a46-bd1a-151e0bc6b1d1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:51.492446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:51.493038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:51.493322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:51.493596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:51.557631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:51.559588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:51.604986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:51.654663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 560/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:09:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:09:51.662477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:51.725648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:51.762476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77815-5963-4fd4-ba59-f898490020f4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:51.795813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:51.796432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:51.796716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:51.796992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] PATH_INFO: `/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:51.798134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b5186e9d514ce9bca6536139f8fb1c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:51.799563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5306d-c21e-4fc1-9f3d-683c4229b283 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:51.801198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 569/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:09:51 2022] GET /identity/v3/projects/e9b5186e9d514ce9bca6536139f8fb1c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:51.807224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:51.902177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5d0-7061-431c-978c-eaf7c1404297 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:51.937341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:51.937942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:51.938225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:51.938501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:51.939218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] RBAC: Adding query filter params (role.id=7647dfe2bb394802a35379f976d57e94, scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c, user.id=0fcb37faf8f14a0e97e6572b6624ead3) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:09:51.939530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:51.940975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fcb-c4ca-4b35-a553-a2e09b5b6c56 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:51.946729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 561/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:09:51 2022] GET /identity/v3/role_assignments?user.id=0fcb37faf8f14a0e97e6572b6624ead3&scope.project.id=e9b5186e9d514ce9bca6536139f8fb1c&role.id=7647dfe2bb394802a35379f976d57e94 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:54.084565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be215e5-bf03-4809-aa0a-678868dcc246 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:54.085091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be215e5-bf03-4809-aa0a-678868dcc246 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:54.085091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be215e5-bf03-4809-aa0a-678868dcc246 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:54.086426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 570/1131] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:54.091723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:54.092138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:54.092280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:54.110206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:54.111374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:54.118442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:54.175176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 562/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:54.441981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be215e5-bf03-4809-aa0a-678868dcc246 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:54.539878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be215e5-bf03-4809-aa0a-678868dcc246 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:54.568209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:54.572961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:54.573579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:54.573884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:54.574158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:54.575254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:54.576723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625556-69d2-4c2e-9489-77b3d5d3a92c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:54.586780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 571/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:54.586919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:54.679578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1ba64-d441-42ef-8e9b-ddd6ca34d80c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:54.712778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:54.713332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:54.713618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:54.713896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:54.714585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:54.716028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e1252-43c6-4d5e-9596-37e08b9e9397 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:54.729872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:54.812917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 563/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:09:54 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:56.976303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584539d-dee1-4582-9f35-11300249a908 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:56.976748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584539d-dee1-4582-9f35-11300249a908 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:56.976838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584539d-dee1-4582-9f35-11300249a908 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:56.978197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 572/1135] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:56.983493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:56.983898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:56.984095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:57.002180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:57.003350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:57.067520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 564/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:09:57.336274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584539d-dee1-4582-9f35-11300249a908 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:57.435242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584539d-dee1-4582-9f35-11300249a908 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:57.468010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:57.468626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:57.468908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:57.469185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:57.470293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:57.471746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db76ed2-a6c0-4885-8442-5a670bf00da8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:57.473274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 573/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:09:57.478940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:09:57.575770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918c2e7-81f6-4a5a-a088-f32350347abd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:09:57.598299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:09:57.609006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:09:57.616697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:57.616697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:57.616697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:57.616697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:09:57.616697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459ecfb-6d69-4a1a-bea7-1c2fa3d59261 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:09:57.704052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 565/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:09:57 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:09:59.862781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07284-210c-488c-8f24-05df49454719 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:59.863312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07284-210c-488c-8f24-05df49454719 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:59.863432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07284-210c-488c-8f24-05df49454719 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:59.864712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 574/1139] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:09:59.870132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:09:59.871027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:09:59.871176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:09:59.889735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:09:59.890913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:09:59.955530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 566/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:09:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:00.225445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07284-210c-488c-8f24-05df49454719 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:00.291693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:00.324426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07284-210c-488c-8f24-05df49454719 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:00.357644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:00.358261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:00.358546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:00.358826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:00.368646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:00.373929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:00.375014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:00.375948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 575/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:00.381078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:00.478388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507d794-a793-4a23-97bb-2467a244f328 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:00.488483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:00.511752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:00.512343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:00.512623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:00.512894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:00.513544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:00.513846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:00.515375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:00.523361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 567/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:00.529248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:00.626144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66db38-26f9-4619-9bb8-c4fff91165c8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:00.652206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:00.659973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:00.660594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:00.662975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:00.662975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] PATH_INFO: `/v3/users/b9e80308d06b4903858ed56bdfa09a9c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:00.674899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] RBAC: Authorizing `identity:get_user(user_id=b9e80308d06b4903858ed56bdfa09a9c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:00.676358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:00.677928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 576/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:00 2022] GET /identity/v3/users/b9e80308d06b4903858ed56bdfa09a9c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:00.684030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:00.783597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b20895-89f9-41c5-ae0d-c19c212221b2 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:00.816121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6a72-e655-4009-af09-68517c565851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:00.816710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6a72-e655-4009-af09-68517c565851 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:00.816992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6a72-e655-4009-af09-68517c565851 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:00.817267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6a72-e655-4009-af09-68517c565851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:00.856200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:00.881323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d6a72-e655-4009-af09-68517c565851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:00.882841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d6a72-e655-4009-af09-68517c565851 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:00.978091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 568/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:10:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:10:00.985699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:01.082510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5c543-878e-4853-8ce8-9448ae63edd7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:01.115039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:01.115683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:01.115974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:01.116251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] PATH_INFO: `/v3/projects/76063284df5c4d928670b1bfc572679a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:01.117358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] RBAC: Authorizing `identity:get_project(project_id=76063284df5c4d928670b1bfc572679a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:01.118791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb448c-1e1a-4f75-aed8-90bdbb1f3d35 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:01.121067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 577/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:00 2022] GET /identity/v3/projects/76063284df5c4d928670b1bfc572679a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:01.127255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6a72-e655-4009-af09-68517c565851 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:01.193817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:01.223127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6a72-e655-4009-af09-68517c565851 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:01.255923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:01.256508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:01.256790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:01.257064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:01.257772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=76063284df5c4d928670b1bfc572679a, user.id=b9e80308d06b4903858ed56bdfa09a9c) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:01.258080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:01.259476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0693e-810b-4ea9-ba80-234e3cfc44d0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:01.264703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 569/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:10:01 2022] GET /identity/v3/role_assignments?user.id=b9e80308d06b4903858ed56bdfa09a9c&scope.project.id=76063284df5c4d928670b1bfc572679a&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:03.379884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c420-5477-48c3-ada9-e42948037bbb None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:03.380417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c420-5477-48c3-ada9-e42948037bbb None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:03.380417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c420-5477-48c3-ada9-e42948037bbb None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:03.381772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 578/1147] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:03.387025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:03.387418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:03.387620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:03.405997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:03.407164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:03.470825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 570/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:03.738392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c420-5477-48c3-ada9-e42948037bbb None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:03.793827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:03.837413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c420-5477-48c3-ada9-e42948037bbb None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:03.870245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:03.870856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:03.871136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:03.871409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:03.881246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:03.886559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:03.887723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:03.888575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 579/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:03.893608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:03.956873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:03.989917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59979961-c494-4762-a9d1-3119f62dce04 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:04.022252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:04.022848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:04.023132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:04.023409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:04.024093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:04.024399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:04.025748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:04.034693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 571/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:04.040628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:04.137864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c40ac-1c8c-4550-aabf-54a59267b346 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:04.151245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:04.171669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:04.172388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:04.172780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:04.173062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] PATH_INFO: `/v3/users/b9e80308d06b4903858ed56bdfa09a9c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:04.174161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] RBAC: Authorizing `identity:get_user(user_id=b9e80308d06b4903858ed56bdfa09a9c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:04.175591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:04.177235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 580/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:04 2022] GET /identity/v3/users/b9e80308d06b4903858ed56bdfa09a9c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:04.182885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:04.277923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bb9c1-3299-4abc-9770-3ef15dd4269d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:04.303773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:04.311222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:04.311875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:04.312160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:04.312436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] PATH_INFO: `/v3/projects/76063284df5c4d928670b1bfc572679a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:04.313541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] RBAC: Authorizing `identity:get_project(project_id=76063284df5c4d928670b1bfc572679a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:04.314997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126a276-ec8d-4832-8c41-99f126ef845a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:04.316603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 572/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:04 2022] GET /identity/v3/projects/76063284df5c4d928670b1bfc572679a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:04.322496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:04.418714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0cd64-5fa6-4e4b-9282-bc6cf36d8f8c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:04.452942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:04.453586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:04.453867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:04.454139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] PATH_INFO: `/v3/projects/76063284df5c4d928670b1bfc572679a/users/b9e80308d06b4903858ed56bdfa09a9c/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:04.456158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] RBAC: Authorizing `identity:create_grant(project_id=76063284df5c4d928670b1bfc572679a, user_id=b9e80308d06b4903858ed56bdfa09a9c, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:04.457568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943867-6373-48f8-853a-796d39158ac1 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:04.538512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 581/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:10:04 2022] PUT /identity/v3/projects/76063284df5c4d928670b1bfc572679a/users/b9e80308d06b4903858ed56bdfa09a9c/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:10:06.645301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd716-fc25-4f60-95ee-c64357cc4b77 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:06.645835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd716-fc25-4f60-95ee-c64357cc4b77 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:06.645835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd716-fc25-4f60-95ee-c64357cc4b77 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:06.647193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 573/1154] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:06.652811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:06.653181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:06.653364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:06.672112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:06.673269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:06.706872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:06.737499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 582/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:07.004632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcd716-fc25-4f60-95ee-c64357cc4b77 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.102619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcd716-fc25-4f60-95ee-c64357cc4b77 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:07.135489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:07.136668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:07.136949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:07.137223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:07.146919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:07.151719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:07.152866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:10:07.153878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 574/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:07.159069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.253840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96477b-72a3-40b8-9353-11b6ccdaff46 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:07.287047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:07.287634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:07.287940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:07.288213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:07.288868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:07.289165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:07.290516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:07.298751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 583/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:07.304505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.358638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:07.400886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b30e0-e251-4201-86d3-5ae3e30b4396 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:07.433898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:07.434512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:07.434794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:07.435148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] PATH_INFO: `/v3/users/b9e80308d06b4903858ed56bdfa09a9c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:07.436241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] RBAC: Authorizing `identity:get_user(user_id=b9e80308d06b4903858ed56bdfa09a9c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:07.437655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:07.439287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 575/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/users/b9e80308d06b4903858ed56bdfa09a9c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:07.445138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.479091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:07.544248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150213ad-3c6c-4f16-bde6-4a67afec3a30 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:07.576806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:07.577400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:07.577680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:07.577954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:07.643526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:07.644981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:07.676852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:07.741034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 584/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:10:07.747475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.844309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e84d2-d1f1-4e43-bfd3-af156731b89e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:07.866091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:07.876639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:07.877245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:07.877525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:07.877798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] PATH_INFO: `/v3/projects/76063284df5c4d928670b1bfc572679a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:07.878904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] RBAC: Authorizing `identity:get_project(project_id=76063284df5c4d928670b1bfc572679a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:07.880342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9602400-d1f8-49d6-a993-c45416471f8f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:07.881933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 576/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/projects/76063284df5c4d928670b1bfc572679a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:07.888077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:07.990143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9423f-d037-453e-b4ca-c49ccdf26fc9 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:08.022494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:08.023095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:08.023379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:08.023676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:08.024403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=76063284df5c4d928670b1bfc572679a, user.id=b9e80308d06b4903858ed56bdfa09a9c) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:08.024713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:08.026139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0551-8bee-43b2-a724-21a96b0fe35e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:08.028587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:08.033314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 585/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:10:07 2022] GET /identity/v3/role_assignments?user.id=b9e80308d06b4903858ed56bdfa09a9c&scope.project.id=76063284df5c4d928670b1bfc572679a&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:10.161818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cfd68-2d4e-4464-a2f6-2531ee6d4d4f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:10.162412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cfd68-2d4e-4464-a2f6-2531ee6d4d4f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:10.162412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cfd68-2d4e-4464-a2f6-2531ee6d4d4f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:10.163712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 577/1162] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:10.169631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:10.170057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:10.170189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:10.187972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:10.189087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:10.253145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 586/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:10.519601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cfd68-2d4e-4464-a2f6-2531ee6d4d4f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:10.616745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cfd68-2d4e-4464-a2f6-2531ee6d4d4f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:10.649237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:10.649820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:10.650099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:10.650373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:10.710394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:10.719900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:10.721366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:10.815055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 578/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:10:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:10:10.820968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:10.916220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08111d-b093-41d8-a563-86a74c87d84a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:10.944995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:10.953147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:10.955043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:10.955043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:10.955043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:10.960992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:10.962419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-75fc85c4-9840-4bed-8054-f6ef7ce5ccd8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:10.974011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:10.975066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:10.976147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 587/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:10:10 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:10.981481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:11.069672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:11.077913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5358e1-1a80-4ea9-b7b1-99458bb82f9f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:11.111551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:11.112171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:11.112454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:11.112730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:11.113368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:11.113671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:11.115016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302454a-cc97-45e1-8552-6b53dbd4e8ab None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:11.121903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 579/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 15 19:10:10 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:13.250794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef459b77-cdce-4327-ae59-979e36694c95 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:13.251327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef459b77-cdce-4327-ae59-979e36694c95 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:13.251327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef459b77-cdce-4327-ae59-979e36694c95 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:13.252702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 588/1167] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:13.257831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d501c-5867-491e-ae77-47aa60b5ca7c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:13.258211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d501c-5867-491e-ae77-47aa60b5ca7c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:13.258396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d501c-5867-491e-ae77-47aa60b5ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:13.276639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fd8d501c-5867-491e-ae77-47aa60b5ca7c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:13.277820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fd8d501c-5867-491e-ae77-47aa60b5ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:13.342804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 580/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:13.612497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef459b77-cdce-4327-ae59-979e36694c95 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:13.711188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef459b77-cdce-4327-ae59-979e36694c95 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:13.743762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:13.744345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:13.744627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:13.744902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:13.745570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:13.746925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1abf8-833e-463a-b8ba-27bb1553e7bb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:13.814016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 589/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Thu Sep 15 19:10:13 2022] POST /identity/v3/domains => generated 267 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:10:15.976391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d605d2-4322-4a6a-87b8-b01bca7b3e09 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:15.976856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d605d2-4322-4a6a-87b8-b01bca7b3e09 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:15.977046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d605d2-4322-4a6a-87b8-b01bca7b3e09 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:15.978295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 581/1170] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:15.983717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:15.984148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:15.984269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:16.001958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:16.003119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:16.045612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:16.068026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 590/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:10:16.351415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d605d2-4322-4a6a-87b8-b01bca7b3e09 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:16.435582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:16.451404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d605d2-4322-4a6a-87b8-b01bca7b3e09 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:16.492164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:16.492785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:16.493071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:16.493343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] PATH_INFO: `/v3/domains/886196bac66843278d356a1d8a0e5fba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:16.512139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=886196bac66843278d356a1d8a0e5fba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:16.513562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31081e6-b1ec-41af-8d16-e9b5e27742e7 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:16.515129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 582/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Sep 15 19:10:16 2022] GET /identity/v3/domains/886196bac66843278d356a1d8a0e5fba => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:16.520969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:16.615738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63ea1d-4665-4241-92cb-d6a96e001191 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:16.648912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:16.649512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:16.649793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:16.650069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:16.650704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:16.652698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd17387-1eb1-4834-a2d8-a4103f988abf None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:16.733048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 591/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:10:16 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:10:18.885942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec658911-c484-408b-ad8d-ee0ddb76e781 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:18.886512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec658911-c484-408b-ad8d-ee0ddb76e781 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:18.886512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec658911-c484-408b-ad8d-ee0ddb76e781 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:18.887851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 583/1174] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:18.893236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:18.893662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:18.893801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:18.912104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:18.913281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:18.937488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:18.978121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 592/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:19.243905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec658911-c484-408b-ad8d-ee0ddb76e781 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:19.340867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec658911-c484-408b-ad8d-ee0ddb76e781 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:19.372873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:19.373485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:19.373764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:19.374035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] PATH_INFO: `/v3/domains/886196bac66843278d356a1d8a0e5fba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:19.375123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=886196bac66843278d356a1d8a0e5fba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:19.376554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de604ca-c8a2-48c3-91d2-898bb315b075 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:19.378068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 584/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Sep 15 19:10:19 2022] GET /identity/v3/domains/886196bac66843278d356a1d8a0e5fba => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:19.383633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:19.480236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653ccd6-c5cb-4bb7-80e8-a25c8101d2c3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:19.512899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:19.513440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:19.513720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:19.513994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:19.514632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:19.516079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b40e02-4568-49e2-ad57-95b3bf8cb92c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:19.629636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 593/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:10:19 2022] POST /identity/v3/users => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:10:21.795229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503facc-bc1f-4cf0-bd11-88bae6c08a2a None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:21.795932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503facc-bc1f-4cf0-bd11-88bae6c08a2a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:21.795932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503facc-bc1f-4cf0-bd11-88bae6c08a2a None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:21.797148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 585/1178] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:21.802265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:21.802682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:21.802819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:21.821260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:21.822422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:21.833301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:21.887985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 594/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:22.154496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503facc-bc1f-4cf0-bd11-88bae6c08a2a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:22.170734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:22.255033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503facc-bc1f-4cf0-bd11-88bae6c08a2a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:22.287528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:22.288170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:22.288454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:22.288727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:22.299033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:22.303808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:22.304737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:10:22.305642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 586/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:22.311156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:22.405388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5c8d6-a2d7-40de-92c3-e66109c9bba1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:22.437730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:22.438314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:22.438592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:22.438861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:22.439508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:22.439837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:22.441180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:22.448803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 595/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:22.454819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:22.528016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:22.551885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3033d8-4740-4622-b694-5cb59197785e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:22.584541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:22.585166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:22.585450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:22.585727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] PATH_INFO: `/v3/users/ff3398623c714e7788948329078ae002` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:22.598982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] RBAC: Authorizing `identity:get_user(user_id=ff3398623c714e7788948329078ae002)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:22.600441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:22.602033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 587/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:22 2022] GET /identity/v3/users/ff3398623c714e7788948329078ae002 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:22.607845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:22.618901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:22.705016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f456a-2348-4584-9487-571c5df3ea36 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:22.737291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:22.737877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:22.738154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:22.738423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:22.802745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:22.804205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:22.813353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:22.903856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 596/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:10:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:10:22.910231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:23.006148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f6dc2-ff87-421c-afb1-5e9ce61c058e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:23.034691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:23.039344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:23.039987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:23.040271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:23.040545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] PATH_INFO: `/v3/projects/1c00346aaafc4a4f9563eaf26bca658f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:23.041668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] RBAC: Authorizing `identity:get_project(project_id=1c00346aaafc4a4f9563eaf26bca658f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:23.043119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0633b0-c183-4ab8-ba33-0ce8707518ad None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:23.044740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 588/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:22 2022] GET /identity/v3/projects/1c00346aaafc4a4f9563eaf26bca658f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:23.050797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:23.148561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9fa53-937b-4636-abb5-748bf8f2ecce None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:23.157853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:23.182828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:23.183733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:23.184016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:23.184290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:23.185005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=1c00346aaafc4a4f9563eaf26bca658f, user.id=ff3398623c714e7788948329078ae002) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:23.185311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:23.186734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a12be9-ee3d-4bb8-8892-2c543a40188d None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:23.192286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 597/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:10:23 2022] GET /identity/v3/role_assignments?user.id=ff3398623c714e7788948329078ae002&scope.project.id=1c00346aaafc4a4f9563eaf26bca658f&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:25.296306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee001-15d0-4af0-bdd8-46eab03a7061 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:25.296845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee001-15d0-4af0-bdd8-46eab03a7061 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:25.296845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efee001-15d0-4af0-bdd8-46eab03a7061 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:25.298190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 589/1186] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:25.303296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:25.303736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:25.303924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:25.321524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:25.322688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:25.387596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 598/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:25.653327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efee001-15d0-4af0-bdd8-46eab03a7061 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:25.749984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efee001-15d0-4af0-bdd8-46eab03a7061 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:25.783057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:25.783701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:25.783989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:25.784265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:25.794037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:25.798804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:25.799930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:10:25.800842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 590/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:25.805724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:25.901556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4ec9-6709-4534-b7c0-a548951a61b1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:25.903008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:25.935323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:25.935961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:25.936238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:25.936512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:25.937166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:25.937468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:25.938825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:25.947148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 599/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:25.952571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:26.009695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:26.049850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69060-7b6b-4bd0-8f2c-f267e23c274e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:26.081151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:26.081753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:26.082033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:26.082310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] PATH_INFO: `/v3/users/ff3398623c714e7788948329078ae002` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:26.083399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] RBAC: Authorizing `identity:get_user(user_id=ff3398623c714e7788948329078ae002)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:26.084838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:26.086373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 591/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:25 2022] GET /identity/v3/users/ff3398623c714e7788948329078ae002 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:26.092702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:26.189629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf0d8-e215-47dd-a65a-4ca76221b534 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:26.221806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:26.222421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:26.222702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:26.222975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] PATH_INFO: `/v3/projects/1c00346aaafc4a4f9563eaf26bca658f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:26.224106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] RBAC: Authorizing `identity:get_project(project_id=1c00346aaafc4a4f9563eaf26bca658f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:26.225531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73d2a-ed60-4e6f-92cf-29f167dcfb15 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:26.227131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 600/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:26 2022] GET /identity/v3/projects/1c00346aaafc4a4f9563eaf26bca658f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:26.233529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:26.334799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe011-ca49-4310-b505-27fec1231017 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:26.360253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:26.367748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:26.368396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:26.368680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:26.370244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] PATH_INFO: `/v3/projects/1c00346aaafc4a4f9563eaf26bca658f/users/ff3398623c714e7788948329078ae002/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:26.371443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] RBAC: Authorizing `identity:create_grant(project_id=1c00346aaafc4a4f9563eaf26bca658f, user_id=ff3398623c714e7788948329078ae002, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:26.372886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd2197-46b1-40e0-8881-52bf02bff2db None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:26.440658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 592/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:10:26 2022] PUT /identity/v3/projects/1c00346aaafc4a4f9563eaf26bca658f/users/ff3398623c714e7788948329078ae002/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:10:28.545778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8351c-8f98-4e43-be85-5014d54b6235 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:28.546310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8351c-8f98-4e43-be85-5014d54b6235 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:28.546310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8351c-8f98-4e43-be85-5014d54b6235 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:28.547695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 601/1193] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:10:28.552875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:28.553257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:28.553432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:28.571691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:10:28.572906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:10:28.637530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 593/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:10:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:10:28.905960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8351c-8f98-4e43-be85-5014d54b6235 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:28.937441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:29.006033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8351c-8f98-4e43-be85-5014d54b6235 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.038322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.038937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.039219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.039491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.050197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:10:29.054880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:10:29.056004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:10:29.057592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 602/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:10:29.062531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:29.157983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccace4-c5c8-4ab7-8315-f4d3e3973dfe None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.175454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:29.192313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.192918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.193200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.193480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.194123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:29.194430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:29.195812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:29.204673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 594/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:10:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:29.210750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:29.299282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:29.308357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c165-effd-432e-b07b-9307213a7cb7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.340496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.341107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.341391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.341666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] PATH_INFO: `/v3/users/ff3398623c714e7788948329078ae002` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.342766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] RBAC: Authorizing `identity:get_user(user_id=ff3398623c714e7788948329078ae002)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:29.344227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:29.345777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 603/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 15 19:10:29 2022] GET /identity/v3/users/ff3398623c714e7788948329078ae002 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:29.351410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:29.447081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d7248-14b6-4054-878d-10430f072c2e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.478348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.479448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.479753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.480026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.525714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:29.545135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:29.546553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:29.640542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 595/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:10:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:10:29.646373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:29.742763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eacb44-7cf3-4cbe-8f22-265ce39ff939 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.774374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.774983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.775263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.776100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] PATH_INFO: `/v3/projects/1c00346aaafc4a4f9563eaf26bca658f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.777199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] RBAC: Authorizing `identity:get_project(project_id=1c00346aaafc4a4f9563eaf26bca658f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:29.778606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb472dc6-d695-49bd-9e30-44c4ade78ead None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:29.780338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 604/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 15 19:10:29 2022] GET /identity/v3/projects/1c00346aaafc4a4f9563eaf26bca658f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:29.786455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:10:29.863107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:29.881639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a139f01-5bfc-408a-9c4f-723fb6708912 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:10:29.915006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:10:29.915605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:10:29.915921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:10:29.916800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:10:29.917521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=1c00346aaafc4a4f9563eaf26bca658f, user.id=ff3398623c714e7788948329078ae002) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:10:29.917835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:10:29.919240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f398b4e-8c61-40b5-8e45-31c1976c1c77 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:10:29.925116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 596/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:10:29 2022] GET /identity/v3/role_assignments?user.id=ff3398623c714e7788948329078ae002&scope.project.id=1c00346aaafc4a4f9563eaf26bca658f&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:10:39.770432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:10:39.922564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=28 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:43.222519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fa7db-4ea3-45bd-902d-2ca638d23ea6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:43.223063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fa7db-4ea3-45bd-902d-2ca638d23ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:43.223063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fa7db-4ea3-45bd-902d-2ca638d23ea6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:43.224444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 605/1201] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:11:43.229662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:43.234071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:43.234071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:43.248847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:11:43.249997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:11:43.315263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 597/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:11:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:11:43.597713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fa7db-4ea3-45bd-902d-2ca638d23ea6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:43.700482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fa7db-4ea3-45bd-902d-2ca638d23ea6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:43.734436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:43.735061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:43.735352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:43.735613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:43.745769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:43.747212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:43.761076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:43.762850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:43.763962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:43.763962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:43.763962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:43.763962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 606/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:11:43.770813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:43.866312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b6033-1c38-40a7-877c-dcfe54b44360 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:43.894082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:43.899604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:43.900232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:43.900515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:43.900789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:43.901424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:43.901728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:43.903123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4974f4-daa6-444f-afee-aae232989e40 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:43.914406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 598/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:43.920114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:43.972755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:44.019105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe26b6-9bfd-420a-b3f6-92e627a8b039 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:44.052481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:44.053088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:44.053372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:44.053646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:44.054314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:44.055759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3f7d-0825-4c90-9697-9e304ae1a911 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:44.170822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 607/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 15 19:11:43 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:11:46.354443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df664-eee4-4bbf-8261-42eada9cc80f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:46.355058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df664-eee4-4bbf-8261-42eada9cc80f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:46.355058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df664-eee4-4bbf-8261-42eada9cc80f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:46.356900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 599/1206] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:11:46.362292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:46.362706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:46.362851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:46.380802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:11:46.381983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:11:46.447235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 608/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:11:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:11:46.723125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df664-eee4-4bbf-8261-42eada9cc80f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:46.824668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df664-eee4-4bbf-8261-42eada9cc80f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:46.857298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:46.857914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:46.858198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:46.858473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:46.868667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:11:46.874261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:46.875426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:46.876410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 600/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:11:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:46.881517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:46.920458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:46.979000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83323ed-b1d4-42d2-bf5b-e40da4bab24b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.013431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.014026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.014303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.014572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.015220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:47.015520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.016903 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.024752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 609/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:47.031134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.080830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:47.127889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520add-e994-444a-8e3a-f92454740151 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.159793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.160432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.160708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.160978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.171492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.172967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:47.187156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:47.188542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:47.189752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:47.189752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:47.189752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:47.189752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 601/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:47.194052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.288145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:47.291100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02f5ee-7a68-467d-921a-048f559e3554 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.324596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.325202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.325484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.325758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.326392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:47.326696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.328092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.339543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 610/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:47.345262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.444287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ddcb0-4856-4074-abf4-0194f926ce1f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.445761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:47.477232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.478560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.478560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.478560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.486016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.487432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:11:47.499301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:47.500291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:47.501292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 602/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:47.506343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.606946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed8954-c391-41b3-b16f-21043d8f6475 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.639357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.639980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.640263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.640537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.641257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:47.641563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.642982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.660342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 611/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:47.666178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.764311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f69-5c92-467a-affb-efc81d7f90ab None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:47.797697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:47.798289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:47.798575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:47.798849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:47.827568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:47.864965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:47.866335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:47.960426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 603/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:11:47.966858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:47.985118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:48.063426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e241cb-741e-4f93-a05d-dc576e0833bb None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:48.098520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:48.099149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:48.099434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:48.099734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:48.109718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:48.111158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:48.125480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:48.126576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:48.131419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:48.131419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:48.131419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:48.131419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 612/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:48.132284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:48.193110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:48.230897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a69e-bc02-4a81-8d8c-277ea71bb069 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:48.263108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:48.263729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:48.264010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:48.264282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:48.264949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:48.265251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:48.266640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:48.279034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 604/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:48.285097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:48.352878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:48.382984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc58cb-dc59-405c-a624-0bb01122d568 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:48.416503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:48.417121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:48.417404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:48.417672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:48.424366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:48.427727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:11:48.438064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:48.438978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:48.439951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 613/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:48.445310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:48.543572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f724c8-a1f3-4fcd-9136-60eca94d37df None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:48.566234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:48.577923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:48.578527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:48.578811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:48.579086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:48.579801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:48.580110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:48.581519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f0f80-4df2-4471-bba4-b1395debac42 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:48.592813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 605/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:48.598926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:48.697798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c61eb-02f7-47cf-9a3c-0f8800f8a1b8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:48.725265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:48.731494 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:48.732119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:48.732448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:48.732727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:48.733442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=bc38e83dcf4b4cd794bbc5c550d7fc05) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:48.733749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:48.735170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ca27e-d45b-43b4-b707-3977fe6d96ad None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:48.740495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 614/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:11:48 2022] GET /identity/v3/role_assignments?user.id=bc38e83dcf4b4cd794bbc5c550d7fc05&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:50.850363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc352-5e5a-40a9-911f-af4e368d0b2f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:50.850899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc352-5e5a-40a9-911f-af4e368d0b2f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:50.850899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc352-5e5a-40a9-911f-af4e368d0b2f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:50.852403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 606/1220] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:11:50.857741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:50.858927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:50.858927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:50.876926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:11:50.878141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:11:50.945506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 615/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:11:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:11:51.220139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc352-5e5a-40a9-911f-af4e368d0b2f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:51.319590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc352-5e5a-40a9-911f-af4e368d0b2f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:51.352498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:51.353113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:51.353397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:51.353674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:51.364388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:11:51.369143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:51.370252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:51.371464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 607/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:11:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:51.376426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:51.474965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbe17c-c6db-48e8-a6de-e3c1ca6aae59 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:51.498225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:51.508598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:51.509228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:51.509510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:51.509784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:51.510445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] RBAC: Adding query filter params (name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:51.510752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:51.512203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:51.519885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 616/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:11:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:51.525705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:51.570560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:51.625057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a91b-4b0e-429c-a0e9-8d3ed976da8b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:51.660149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:51.660761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:51.661042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:51.661314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:51.671168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:51.675402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:51.686838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:51.687987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:51.688980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:51.688980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:51.688980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:51.688980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 608/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:51.693871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:51.791741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2146d4-295c-442b-a780-1cb0e5a6b898 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:51.823333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:51.824485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:51.824763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:51.825033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:51.825660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:51.825959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:51.827313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:51.838547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 617/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:51.845526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:51.937568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:51.944011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078947f-c960-44bd-b043-5e4df9d6e773 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:51.976819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:51.977435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:51.977713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:51.977983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:51.984748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:51.986168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:11:51.998272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:51.999462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:52.000681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 609/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:52.005532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.056313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:52.104321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a49ef6-a3f4-4cde-8781-06ab801eb774 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.137856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.138437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.138719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.138994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.139740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:52.140050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.141467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:52.158185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 618/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:11:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:52.164075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.262338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb1b7d-4e41-4ee2-9ee5-a11210ea814c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.294703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.295312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.295595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.295897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.305996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.307469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:52.321498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:52.322627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:52.324472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:52.324472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:52.324472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:52.324472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 610/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:52.331399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.426124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd43658-e8ca-4e0c-b172-56b5940196df None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.428421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:52.459778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.460377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.460660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.460936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.461571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:52.461876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.463232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:52.473981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 619/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:52.480626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.489911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:52.579662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969311ed-17fe-4548-a94c-fdc4f622adbb None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.612910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.613531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.613813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.614087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.620473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.622442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:11:52.633467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:52.638219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:52.639344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 611/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:11:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:52.640685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.739923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528df8b-97b1-4170-9927-92f4f5f47990 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.773233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.773832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.774111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.774386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.775074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:52.775378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.777412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493604e-8624-4491-a869-2e632306ae67 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:52.789339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 620/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:11:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:52.796524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:52.860702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:52.893577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee11cc3-b308-499c-a321-b8d7e3c020dc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:52.927083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:52.927437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:52.927759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:52.928039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/bc38e83dcf4b4cd794bbc5c550d7fc05/roles/64fdd85e215d407c9acc7945a28b8686` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:52.942576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=bc38e83dcf4b4cd794bbc5c550d7fc05, role_id=64fdd85e215d407c9acc7945a28b8686)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:52.944049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabe260-6ef8-4978-9459-bbf38419de4e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:53.078721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 612/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:11:52 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/bc38e83dcf4b4cd794bbc5c550d7fc05/roles/64fdd85e215d407c9acc7945a28b8686 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:11:55.199539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50386a-6b8b-4a4a-9469-f858ab1d2bbc None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:55.200106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50386a-6b8b-4a4a-9469-f858ab1d2bbc None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:55.200106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50386a-6b8b-4a4a-9469-f858ab1d2bbc None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:55.201380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 621/1233] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:11:55.206754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:55.207172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:55.207340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:55.225255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:11:55.226429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:11:55.292890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 613/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:11:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 19:11:55.574279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50386a-6b8b-4a4a-9469-f858ab1d2bbc None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:55.599333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:55.675102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50386a-6b8b-4a4a-9469-f858ab1d2bbc None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:55.708613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:55.709231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:55.709515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:55.709791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:55.720309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:11:55.725069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:55.726159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:55.727207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 622/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 15 19:11:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:11:55.732882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:55.780809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:55.834638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19972001-6d4e-45e7-9ba8-96e945f3d551 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:55.868280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:55.868880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:55.869168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:55.869444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:55.870098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:55.870542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:55.871927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:55.880065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 614/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 15 19:11:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:55.885962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:55.969746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:55.985471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a93cb-bd7d-4142-be01-06b0847d86e7 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.017410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.018025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.018301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.018569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.028621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.030046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:56.044873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:56.046978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:56.047907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:56.047907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:56.047907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:56.047907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 623/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:56.052066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:56.152362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ba0f-b417-40c5-868c-88d054a0f2c3 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.161150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:56.184929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.185532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.185844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.186169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.186800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:56.187106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.188505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.201742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 615/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:56.207800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:56.306049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cca4e-3f73-4de0-bd84-4ffcb29b068f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.340262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.340885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.341170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.341449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.344099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:56.349002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.350446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:11:56.361335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:56.363869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:11:56.364851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 624/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:56.368419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:56.469544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4662fa-0f5b-45f6-9786-80df3118e759 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.503890 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.504492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.504773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.505047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.505810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:56.506116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.507529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.524627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 616/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:11:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:56.531503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:56.633726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d3f1-0397-4974-b797-5e38c1782c75 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.667090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.667688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.667972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.668245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.728353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:56.734651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.736150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.834579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 625/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:11:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:11:56.840509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:56.867134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:56.938889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ab24-1cea-4bf7-a8d6-0e3449345746 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:56.970875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:56.971497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:56.971814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:56.972093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:56.982328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:56.983713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:11:56.998600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:57.000298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:11:57.001272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:11:57.001272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:11:57.001272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:57.001272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 617/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:57.006208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:57.099237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:57.104909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef9d8-57e2-4b77-ab27-5194b3cc718c None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:57.138388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:57.144982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:57.153452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 626/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:57.159578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:57.238623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:57.258997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69bd29-82d2-40da-8fed-37f66a08aae6 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:57.292908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:57.293509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:57.293791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:57.294064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:57.300374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:57.301807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:11:57.313154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:11:57.314294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:11:57.315380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 618/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:11:57.320132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:57.417664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edfb11-f92a-479b-8d59-03d0450364c4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:57.450558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:57.451187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:57.451474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:57.451778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:57.452451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:57.452755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:57.454159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160f48-e440-4d36-be3b-fe6dbb0714e0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:57.465814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 627/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:11:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:57.472010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:11:57.569876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd490a4-0441-45a7-a512-dc9dc61b985a None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:11:57.603404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:11:57.604022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:57.604302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:57.604874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:57.605371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] RBAC: Adding query filter params (role.id=64fdd85e215d407c9acc7945a28b8686, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=bc38e83dcf4b4cd794bbc5c550d7fc05) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:11:57.605673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:11:57.607176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f2ac-ac25-4941-b4f9-f8828afed543 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:11:57.612779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 619/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:11:57 2022] GET /identity/v3/role_assignments?user.id=bc38e83dcf4b4cd794bbc5c550d7fc05&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=64fdd85e215d407c9acc7945a28b8686 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:11:59.789823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d157a0-2127-45e2-a3df-6b0ffca4b356 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:59.790419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d157a0-2127-45e2-a3df-6b0ffca4b356 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:59.790419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d157a0-2127-45e2-a3df-6b0ffca4b356 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:59.791812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 628/1247] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:11:59.798056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:11:59.798465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:11:59.798633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:11:59.804870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:11:59.816692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:11:59.817898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:11:59.881545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 620/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:11:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:00.154606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d157a0-2127-45e2-a3df-6b0ffca4b356 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.172715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:00.253361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d157a0-2127-45e2-a3df-6b0ffca4b356 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:00.287615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:00.288289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:00.288568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:00.288842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:00.300304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:12:00.305035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:00.306226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:00.307296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 629/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:00.312761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.409961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6246e-32c6-46f3-a40e-98fcf0eb76d6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:00.438650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:00.443437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:00.444070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:00.444354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:00.444628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:00.445279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:00.445582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:00.446922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:00.455363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 621/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:00.461416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.538418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:00.560849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98a3fb-3ab8-4a46-a071-0dfaeb588258 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:00.593252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:00.593887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:00.594174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:00.594454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:00.605397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:00.606839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:00.621376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:00.622500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:00.623475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:00.623475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:00.623475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:00.623475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 630/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:00.628254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.724464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8e54-5816-4555-9c72-66cc6f910384 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:00.757892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:00.758487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:00.758767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:00.759040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:00.759697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:00.760087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:00.761451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:00.772713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 622/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:00.778581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.876959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d422-0522-4148-a103-1bf08894cfc8 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:00.902312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:00.910233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:00.910864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:00.911143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:00.911412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:00.919511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:00.921236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:12:00.932476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:00.934150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:00.935486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 631/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:00.939993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:00.995897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:01.038984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca61b-43b0-4b49-a827-fc6f5bd448d1 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:01.071441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:01.072081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:01.072392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:01.072667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:01.073461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:01.073773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:01.075185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:01.092084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 623/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:12:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:01.098013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:01.198009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a57a35-b8ba-44cc-9d82-91218429b498 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:01.231981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:01.232614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:01.232894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:01.233166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:01.287223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:01.299504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:01.300978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:01.396519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 632/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:12:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:12:01.403059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:01.499917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577345-3dc9-4e8b-98a7-0ef1bf50dd21 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:01.513153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:01.533875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:01.534532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:01.534811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:01.535081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:01.545093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:01.546493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:01.560815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:01.561774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:01.562795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:01.562795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:01.562795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:01.562795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 624/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:01.567639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:01.653610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:01.665360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c227a-e094-419f-a754-06d5f4576d97 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:01.697480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:01.698679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:01.698963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:01.699234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:01.699888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:01.700185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:01.701538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:01.712311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 633/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:01.718781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:01.817092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fba56-4c6a-44d6-9529-6e9a92fd3cca None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:01.849434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:01.850048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:01.850328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:01.850599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:01.857116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:01.858534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:12:01.869282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:01.870916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:01.871952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 625/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:12:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:01.877052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:01.977529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb005-33ce-4578-beb6-dd61e5509c1f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:02.011348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:02.011971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:02.012250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:02.012518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:02.013198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:02.013502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:02.014971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ca19-8579-4cee-8351-4992924a0863 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:02.027295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 634/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:12:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:02.033691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:02.059476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:02.131933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce28791-3ef6-4141-92b9-34def8be1639 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:02.165997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:02.166591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:02.166886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:02.167150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:02.167895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=bc38e83dcf4b4cd794bbc5c550d7fc05) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:02.168203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:02.169605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bada9-e16b-4673-abf4-0e89507d7cf8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:02.174980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 626/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:12:02 2022] GET /identity/v3/role_assignments?user.id=bc38e83dcf4b4cd794bbc5c550d7fc05&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:04.281705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e244b-f6e0-44db-a6de-947db6210a52 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:04.282231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e244b-f6e0-44db-a6de-947db6210a52 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:04.282369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e244b-f6e0-44db-a6de-947db6210a52 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:04.283769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 635/1261] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:04.289012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:04.289450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:04.289545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:04.307094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:04.308844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:04.373006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 627/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:04.644224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e244b-f6e0-44db-a6de-947db6210a52 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:04.655437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:04.746518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e244b-f6e0-44db-a6de-947db6210a52 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:04.779222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:04.779867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:04.780400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:04.780670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:04.790412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:12:04.795790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:04.803723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:04.804611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 636/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:12:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:04.805322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:04.821845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:04.898998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a402bde-a760-4228-b1bb-4c484ccc9a7c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:04.933735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:04.934340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:04.934622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:04.934896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:04.935552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:04.935883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:04.937229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:04.944995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 628/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:12:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:04.950706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.017552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:05.050534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4d9c1-32ec-442f-a467-274889ff7eac None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.082681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.083298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.083580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.083883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.093964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.095416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:05.110282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:05.113152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:05.114278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:05.114278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:05.114278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:05.114278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 637/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:05.117232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.198973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:05.217458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ab43-fced-4c01-ac1d-419af24c101e None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.249480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.250086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.250368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.250644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.251279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:05.251586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.252992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.265111 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 629/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:05.270851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.368390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc289f-857a-4e00-9986-ba5db19323b4 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.388137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:05.404836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.405766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.406036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.406304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.413858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.415331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:12:05.426972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:05.428226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:05.429429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 638/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:05.434024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.532007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77934aa-1f4b-4bc0-ac44-df53ded0b80f None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.564815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.565440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.565721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.566022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.566742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:05.567053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.568489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.577960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:05.585904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 630/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:05.591991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.692299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2fc35-0b3f-4fa5-8645-e831af00b720 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.724432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.725104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.725379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.725647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.735721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.737147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:05.751131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:05.752320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:05.753420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:05.753420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:05.753420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:05.753420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 639/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:05.758083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.860809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba340ea-086f-4d96-a2d2-f9347a425e73 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:05.897344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:05.897945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:05.898231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:05.898506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:05.899143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:05.899451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:05.900841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:05.912170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 631/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:05.918443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:05.950433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:06.018777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8446-3edf-4773-84ed-39f6d3488eee None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:06.052777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:06.053401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:06.053764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:06.054035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:06.065301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:06.066730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:12:06.077970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:06.082843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:06.084012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 640/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:06.085029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:06.142720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:06.181960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6f53-5adb-4056-b394-8a4275d9c316 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:06.214460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:06.215039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:06.215524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:06.215874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:06.216564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:06.217091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:06.218683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61be4f7-1ec4-439b-ae3d-1e6306a95dd0 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:06.231826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 632/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:12:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:06.238853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:06.329024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:06.337323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7890a0-c1cb-4402-bf5d-9a85a147b827 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:06.369844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:06.370508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:06.370786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:06.371057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] PATH_INFO: `/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/bc38e83dcf4b4cd794bbc5c550d7fc05/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:06.372945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb07dc6930943a98070c1ab5007d449, user_id=bc38e83dcf4b4cd794bbc5c550d7fc05, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:06.374703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7abf50-2ee2-4a8d-9898-89c204765272 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:06.461251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 641/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 15 19:12:06 2022] PUT /identity/v3/projects/dcb07dc6930943a98070c1ab5007d449/users/bc38e83dcf4b4cd794bbc5c550d7fc05/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:12:08.616893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2c15-e455-4a0b-b587-b6cbe2eac676 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:08.617424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2c15-e455-4a0b-b587-b6cbe2eac676 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:08.617562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2c15-e455-4a0b-b587-b6cbe2eac676 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:08.618788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 633/1274] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:08.624635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:08.625044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:08.625222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:08.644075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:08.645336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:08.711551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 642/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:08.984219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe2c15-e455-4a0b-b587-b6cbe2eac676 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.084679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe2c15-e455-4a0b-b587-b6cbe2eac676 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.101711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:09.118221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.118844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.119130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.119460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.129742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 19:12:09.134523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:09.135671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:09.136808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 634/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 15 19:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:09.141788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.220466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:09.238217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90ee8-099d-4a6f-bc95-17e4816e0908 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.271756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.272354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.272635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.272910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.273566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:09.273872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:09.275267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:09.283905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 643/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:09.289749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.387656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7cfa0-e615-4bb0-8967-00441132fa6d None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.421644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.422278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.422584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.422863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.432710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:09.434249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:09.448701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:09.449836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:09.451067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:09.451067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:09.451067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:09.451067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 635/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:09.455599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.555002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2d7d2-6b61-477a-9e90-61fb3b49fc88 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.587268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.587983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.588238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.589005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.589641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:09.589944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:09.592857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:09.602881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 644/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:09.609376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.639484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:09.707047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d2ce2-74a2-4651-9f74-651803023b76 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.739943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.740556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.740837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.741109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.747820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:09.749289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 19:12:09.760971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:09.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:09.763547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 636/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:09.767898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:09.771782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:09.868567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a773a-9495-4ca2-9333-6780c594517e None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:09.899176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:09.899804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:09.900086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:09.900360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:09.905535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:09.905535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:09.905535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:09.920466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 645/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:09.926367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:10.014346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:10.023739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e556-f97f-437a-ba09-85338726cf6c None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:10.057790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:10.058532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:10.058813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:10.059085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:10.123313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:10.124718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:10.213657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:10.224042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 637/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 15 19:12:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 19:12:10.230534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:10.294114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:10.332214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa809f-63e9-4b44-8b31-8d769daaeb6f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:10.365782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:10.366412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:10.366692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:10.366965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:10.377412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:10.378798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 19:12:10.393567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:10.394682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 19:12:10.395814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 19:12:10.395814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 19:12:10.395814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:10.395814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 646/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:10.400555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:10.498298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d782b2-b287-4270-9036-d2de8f5523c4 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:10.531315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:10.531946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:10.532234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:10.532511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:10.533151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:10.533457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:10.534805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:10.547017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 638/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 15 19:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:10.552710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:10.661341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057454e-3f7d-4a34-b0a9-c0b464021fb3 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:10.676058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:10.697724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:10.698411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:10.699517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:10.699517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:10.706504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:10.707995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 19:12:10.720196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:10.726807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: ERROR keystone.server.flask.application Sep 15 19:12:10.727994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 647/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 15 19:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 19:12:10.728586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:10.782576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:10.835593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21626ba-88eb-4369-962c-ed66b4b21804 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:10.872140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:10.872764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:10.873050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:10.873327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:10.874033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:10.874344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:10.875807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fab80-7c2c-4cae-a773-5cf5f4b7b7ca None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:10.888760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 639/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 15 19:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:10.895732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:11.010521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549abbd-0da3-4280-a6e4-edbbcd5df0c2 None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:11.047962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:11.048602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:11.048885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:11.049159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:11.049912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] RBAC: Adding query filter params (role.id=1ae7138219844425af3d0d3f1ef8ce39, scope.project.id=dcb07dc6930943a98070c1ab5007d449, user.id=bc38e83dcf4b4cd794bbc5c550d7fc05) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:11.050224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:11.051710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64795a-d36e-415e-a1c3-41b788dbfa76 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:11.057618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 648/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 15 19:12:10 2022] GET /identity/v3/role_assignments?user.id=bc38e83dcf4b4cd794bbc5c550d7fc05&scope.project.id=dcb07dc6930943a98070c1ab5007d449&role.id=1ae7138219844425af3d0d3f1ef8ce39 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:13.223427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bda3c-3d86-4450-84e3-01d64dc1ffce None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:13.224019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bda3c-3d86-4450-84e3-01d64dc1ffce None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:13.224019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bda3c-3d86-4450-84e3-01d64dc1ffce None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:13.225256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 640/1288] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:13.230369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:13.238039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:13.238039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:13.249348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:13.250522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:13.279441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:13.316413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 649/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:13.584302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bda3c-3d86-4450-84e3-01d64dc1ffce None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:13.684208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bda3c-3d86-4450-84e3-01d64dc1ffce None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:13.694377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:13.716500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:13.717118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:13.717398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:13.717669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:13.718242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:13.719720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:13.728897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:13.730638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:13.731569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 641/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:12:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:13.736642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:13.833653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91aa1f3-71ff-4895-a61d-39fe3f300b9d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:13.867567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:13.868172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:13.868456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:13.868731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:13.869353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:13.869655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:13.871013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8022f8-e214-4678-8f64-97ec281f66b2 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:13.875776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 650/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 15 19:12:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:13.881475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:13.981127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f652-d377-4adb-827a-eb51b28512cc None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:14.014379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:14.014972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:14.015254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:14.015526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] PATH_INFO: `/v3/services` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:14.016172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:14.016475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:14.017914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0a481-c887-4a10-a121-db465dca81e8 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:14.022143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 642/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 15 19:12:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:16.143244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefec6a-5c6b-44c2-afb2-16694f45e12c None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:16.143822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefec6a-5c6b-44c2-afb2-16694f45e12c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:16.143822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefec6a-5c6b-44c2-afb2-16694f45e12c None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:16.145211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 651/1293] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:16.150625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601861-d490-451f-b782-ef7de4977950 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:16.151085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601861-d490-451f-b782-ef7de4977950 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:16.151185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601861-d490-451f-b782-ef7de4977950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:16.169190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-81601861-d490-451f-b782-ef7de4977950 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:16.170384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-81601861-d490-451f-b782-ef7de4977950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:16.199773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:16.235033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 643/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:16.502429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefec6a-5c6b-44c2-afb2-16694f45e12c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:16.532862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:16.602881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefec6a-5c6b-44c2-afb2-16694f45e12c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:16.636753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:16.637352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:16.637635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:16.637911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:16.638555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:16.639987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b01e88-0f6a-4b51-a543-2abe53a383d7 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:16.706625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 652/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 15 19:12:16 2022] POST /identity/v3/services => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:12:18.861432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16e2c7-0649-40c6-906f-5877fd4af0ba None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:18.862008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16e2c7-0649-40c6-906f-5877fd4af0ba None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:18.862008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16e2c7-0649-40c6-906f-5877fd4af0ba None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:18.863474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 644/1296] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:18.868626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:18.869018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:18.869177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:18.887571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:18.888816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:18.960386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 653/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:19.227606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16e2c7-0649-40c6-906f-5877fd4af0ba None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:19.327341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16e2c7-0649-40c6-906f-5877fd4af0ba None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:19.360780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:19.361402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:19.361688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:19.361965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:19.362544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:19.364054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:19.372734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:19.373897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:19.374735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 645/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:12:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:19.380029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:19.398998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:19.476949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ebc6d-5009-4bb2-aa2f-064df277917d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:19.512765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:19.513404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:19.513690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:19.513963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:19.514585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:19.514893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:19.516294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:19.521303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 654/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 15 19:12:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:19.527421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:19.559554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:19.625329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e544ae-cd65-40f1-8a52-95a67e05c43b None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:19.657536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:19.658222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:19.658504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:19.658779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:19.659495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] RBAC: Adding query filter params (interface=public, service_id=613eb26885bf4ee6b61c21a8749a3d22, region_id=RegionOne) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:19.659831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:19.661178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e8af6-d077-4103-8376-e47aac06449c None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:19.666192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 646/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 15 19:12:19 2022] GET /identity/v3/endpoints?service_id=613eb26885bf4ee6b61c21a8749a3d22&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:19.672580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:19.753766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:19.771202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b5da6-f066-44fe-baa5-c9eac58a7f5f None admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:19.804745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:19.805368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:19.805648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:19.805918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:19.806512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:19.807902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3101b-f3ce-4e6c-ab26-150705ac49a0 None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:19.814509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 655/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 15 19:12:19 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:12:21.960457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e7d0-fdef-4474-9b7a-9a21186c6534 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:21.960968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e7d0-fdef-4474-9b7a-9a21186c6534 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:21.961070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e7d0-fdef-4474-9b7a-9a21186c6534 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:21.962350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 647/1302] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:12:21.968133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:21.968544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:21.968695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:21.986544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:12:21.987757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:12:22.059139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 656/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:12:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 19:12:22.327124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e7d0-fdef-4474-9b7a-9a21186c6534 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:22.420202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:22.426656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e7d0-fdef-4474-9b7a-9a21186c6534 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:22.460019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:22.460640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:22.460922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:22.461195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:22.461771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:22.463120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 19:12:22.472247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 19:12:22.473410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application with Lock( Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self._enter() Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application return self.creator() Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application created_value = creator( Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: ERROR keystone.server.flask.application Sep 15 19:12:22.474446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 648/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 15 19:12:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 19:12:22.479432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:22.555140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:22.578839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07297a0-2c2f-4f7a-b714-ad3fb4809ef0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:22.610906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:22.611509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:22.611828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:22.612111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] PATH_INFO: `/v3/services` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:22.612755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:12:22.613062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:22.614453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154c108-9fdd-4cf7-a646-b3e5f60f348b None admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:22.619441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 657/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 15 19:12:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:12:22.625951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:12:22.725077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae102f-732d-45d8-9cee-c9ed0d025e42 None admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:12:22.758021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:12:22.758623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:12:22.758929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:12:22.759215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:12:22.759903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:12:22.767820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980238-ce48-4fd0-ac85-5e5d7082b199 None admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:12:22.849323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 649/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:12:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:12:32.626598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=30 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:12:32.844767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:20.857883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98e0ca-355c-4faf-bde3-d79e83545a26 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:20.858430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98e0ca-355c-4faf-bde3-d79e83545a26 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:20.858430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98e0ca-355c-4faf-bde3-d79e83545a26 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:20.869230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292365-1171-48a2-a5be-1b4bba5255c7 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:20.869230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292365-1171-48a2-a5be-1b4bba5255c7 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:20.869230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292365-1171-48a2-a5be-1b4bba5255c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:20.870834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 658/1307] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:20.952699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-71292365-1171-48a2-a5be-1b4bba5255c7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:20.953921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-71292365-1171-48a2-a5be-1b4bba5255c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:21.075187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 650/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:18:21.419716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:21.419716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:21.419716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:21.419716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 659/1309] 10.222.0.18 () {54 vars in 965 bytes} [Thu Sep 15 19:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:21.420561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826c7f-a1cd-4927-9dd5-412821f6e200 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:21.420561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826c7f-a1cd-4927-9dd5-412821f6e200 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:21.420561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826c7f-a1cd-4927-9dd5-412821f6e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:21.481471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-cc826c7f-a1cd-4927-9dd5-412821f6e200 None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:21.482713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-cc826c7f-a1cd-4927-9dd5-412821f6e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:21.505259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:21.574903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 651/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Sep 15 19:18:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:21.584272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:21.586320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:21.706157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53074e1-bff9-4388-8aa6-e538f4cbe9b5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:21.785958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:21.790520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:21.790520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:21.790520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:21.790520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:21.849427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:21.906922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:21.910795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de7e7-9d16-468b-9717-dc0cf31ac213 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:21.931281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 660/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:24.377686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550391fe-5761-44c3-9b23-00b103bbc2e6 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:24.378174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550391fe-5761-44c3-9b23-00b103bbc2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:24.378284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550391fe-5761-44c3-9b23-00b103bbc2e6 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:24.379570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 652/1312] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:24.385224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfea1e-6ef1-473c-b54f-70acd449c8e1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:24.385628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfea1e-6ef1-473c-b54f-70acd449c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:24.385765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfea1e-6ef1-473c-b54f-70acd449c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:24.460880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1fcfea1e-6ef1-473c-b54f-70acd449c8e1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:24.462092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1fcfea1e-6ef1-473c-b54f-70acd449c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:24.556329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 661/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:24.850388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:24.850675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:24.850985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:24.852885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 653/1314] 10.222.0.18 () {54 vars in 965 bytes} [Thu Sep 15 19:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:24.858393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d7f-2671-47e5-b7a0-96b54b473556 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:24.858678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d7f-2671-47e5-b7a0-96b54b473556 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:24.858950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0772d7f-2671-47e5-b7a0-96b54b473556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:24.871151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:24.936938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b0772d7f-2671-47e5-b7a0-96b54b473556 None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:24.938135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b0772d7f-2671-47e5-b7a0-96b54b473556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:25.012559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 662/1315] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Sep 15 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:25.021323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:25.022870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:25.126896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3354b-a312-4d19-a727-4516bfc65ac8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:25.198122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:25.200558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:25.201175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:25.201459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:25.201730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:25.203472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:25.313972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:25.315316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316b77f-1e95-4d00-9420-6ee2c51da0e1 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:25.334472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 654/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:28.197372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:28.197892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:28.197892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:28.199189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 663/1317] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:28.204386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f08-5a03-488f-af58-c9518668537f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:28.204809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f08-5a03-488f-af58-c9518668537f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:28.204942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f08-5a03-488f-af58-c9518668537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:28.269686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-30197f08-5a03-488f-af58-c9518668537f None None] MFA Rules not processed for user `e7e41ce832594a9d94ee9f12464ad846`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:28.270874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-30197f08-5a03-488f-af58-c9518668537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:28.324130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:28.368721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 655/1318] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:28.656919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:28.658471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:28.761465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdafaa3-3fdd-426b-8c7e-871092d12ae1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:28.764299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:28.833040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:28.833633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:28.833915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:28.834184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:28.835816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:28.944084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:28.945576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ad6e2-dc25-4089-bd66-24c81bbb6226 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:28.963515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 664/1319] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:32.022341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231554e-d523-4761-8160-10a16907ae64 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:32.022865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231554e-d523-4761-8160-10a16907ae64 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:32.022865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231554e-d523-4761-8160-10a16907ae64 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:32.024348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 656/1320] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:32.029475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45c42-f76b-44e7-aca5-977d0255b671 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:32.029849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45c42-f76b-44e7-aca5-977d0255b671 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:32.030033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45c42-f76b-44e7-aca5-977d0255b671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:32.070941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:32.095148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b3b45c42-f76b-44e7-aca5-977d0255b671 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:32.096379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b3b45c42-f76b-44e7-aca5-977d0255b671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:32.190236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 665/1321] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:32.469815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231554e-d523-4761-8160-10a16907ae64 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:32.471404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4231554e-d523-4761-8160-10a16907ae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:32.574820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231554e-d523-4761-8160-10a16907ae64 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:32.646990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:32.647585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:32.647899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:32.648198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:32.649769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:32.657298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:32.758774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:32.760276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235328-ee9d-4a16-82ef-e5b9c97d7898 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:32.778767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 657/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:36.109134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:36.109696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:36.109696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:36.111005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 666/1323] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:36.116545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2b92-29b8-4ff7-b76b-5e23397f2694 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:36.117531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2b92-29b8-4ff7-b76b-5e23397f2694 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:36.117725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2b92-29b8-4ff7-b76b-5e23397f2694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:36.190252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f8cd2b92-29b8-4ff7-b76b-5e23397f2694 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:36.191416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f8cd2b92-29b8-4ff7-b76b-5e23397f2694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:36.256464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:36.299038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 658/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:36.581741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:36.583312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:36.689610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cec37-1b8d-42ca-bcb8-c153c61a64e6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:36.703945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:36.770601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:36.775983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:36.775983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:36.775983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:36.775983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:36.884726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:36.886082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98657d2d-7967-481f-be34-0e0ba9954db9 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:36.904285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 667/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:39.088467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:39.088995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:39.089091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:39.090706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 659/1326] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:39.096073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565008e8-9487-4976-90a0-f90cba2e81b6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:39.096531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565008e8-9487-4976-90a0-f90cba2e81b6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:39.096639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565008e8-9487-4976-90a0-f90cba2e81b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:39.144696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:39.162108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-565008e8-9487-4976-90a0-f90cba2e81b6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:39.163316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-565008e8-9487-4976-90a0-f90cba2e81b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:39.265692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 668/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:39.549575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:39.551166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:39.655972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb230e1-6737-4ed4-b944-9e026a48e278 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:39.731291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:39.731925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:39.732211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:39.732480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:39.734127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:39.751010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:39.851472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:39.853059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5e887-ba09-44d1-953b-8c57555d0537 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:39.871458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 660/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:42.071204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:42.071744 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:42.071863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:42.073525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 669/1329] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:42.078946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ff31-c56c-4bfe-8abd-d3fa2f11a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:42.079346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ff31-c56c-4bfe-8abd-d3fa2f11a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:42.079546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ff31-c56c-4bfe-8abd-d3fa2f11a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:42.146030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f486ff31-c56c-4bfe-8abd-d3fa2f11a1c1 None None] MFA Rules not processed for user `e7e41ce832594a9d94ee9f12464ad846`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:42.147225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f486ff31-c56c-4bfe-8abd-d3fa2f11a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:42.212805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:42.244595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 661/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:18:42.528856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:42.530482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:42.644503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101d602-328b-4051-87eb-bb96a6c50284 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:42.667394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:42.723554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:42.724273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:42.724566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:42.724842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:42.728297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:42.842162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:42.843604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03bb82-eb86-4c84-a984-272f18f7e817 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:42.862009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 670/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:45.440661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:45.441227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:45.441227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:45.442541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 662/1332] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:45.448273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d897-8b1e-4595-bab9-6d5ad5668e45 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:45.448664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d897-8b1e-4595-bab9-6d5ad5668e45 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:45.448834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d897-8b1e-4595-bab9-6d5ad5668e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:45.512021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:45.514594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ec12d897-8b1e-4595-bab9-6d5ad5668e45 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:45.515824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ec12d897-8b1e-4595-bab9-6d5ad5668e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:45.612473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 671/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:45.889169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:45.890722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:45.994907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1f0a-9f09-456e-a47d-f86f8a3fc220 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:46.068172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:46.068769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:46.069055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:46.069335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:46.071437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:46.098130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:46.179812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:46.181168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b34b2-4e8d-4c53-8d9c-324e855ed4b3 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:46.199048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 663/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:48.384012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:48.384526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:48.384641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:48.385914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 672/1335] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:48.391102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84b84-e664-4fb7-b2ec-ebd551a260f5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:48.391523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84b84-e664-4fb7-b2ec-ebd551a260f5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:48.391683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84b84-e664-4fb7-b2ec-ebd551a260f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:48.457576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-00d84b84-e664-4fb7-b2ec-ebd551a260f5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:48.458786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-00d84b84-e664-4fb7-b2ec-ebd551a260f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:48.533245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:48.555856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 664/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:48.839624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:48.841224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:48.943945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a88117-9aea-49df-b5d5-addacdbe3a1a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:48.972681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:49.017197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:49.017818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:49.018097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:49.018371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:49.019993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:49.126275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:49.127624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ff27-b153-4abd-acdb-0e794180a663 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:49.145466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 673/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:51.892646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:51.893223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:51.893223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:51.894556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 665/1338] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:51.900186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775b00-eade-4b5c-a83d-20b7767f0f3e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:51.900587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775b00-eade-4b5c-a83d-20b7767f0f3e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:51.900742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775b00-eade-4b5c-a83d-20b7767f0f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:51.966522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a0775b00-eade-4b5c-a83d-20b7767f0f3e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:51.967748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a0775b00-eade-4b5c-a83d-20b7767f0f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:51.975929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:52.066205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 674/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:52.348022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:52.349623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:52.452737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869b600-9d80-48c1-a535-fc41b899c7cd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:52.524470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:52.525070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:52.525354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:52.525629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:52.527219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:52.563390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:52.638045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:52.639516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c7f7e-4662-421a-8a86-f7d083d997ce service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:52.658090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 666/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:18:57.108145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:57.108684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:57.108790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:57.115855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f2c0-3b6e-4f18-bee7-c4d17cc21cf1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:57.119897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f2c0-3b6e-4f18-bee7-c4d17cc21cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:57.119897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f2c0-3b6e-4f18-bee7-c4d17cc21cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:57.120091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 675/1341] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:18:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:18:57.191432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7441f2c0-3b6e-4f18-bee7-c4d17cc21cf1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:18:57.192835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7441f2c0-3b6e-4f18-bee7-c4d17cc21cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:57.283475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:57.299690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 667/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:18:57.579910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:18:57.581443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:57.686755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b3b3-e13c-4ad6-b572-0c800f12d5a0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:18:57.726011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:18:57.762522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:18:57.763138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:18:57.763422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:18:57.763722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:18:57.765360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:18:57.876530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:18:57.877966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef6ece-0c29-46f6-bee7-12fa7a67f233 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:18:57.896858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 676/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:00.967756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:00.968258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:00.968347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:00.969786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 668/1344] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:00.974830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a845b-6061-45b3-aafb-d50c3f3b9c34 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:00.975236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a845b-6061-45b3-aafb-d50c3f3b9c34 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:00.975460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a845b-6061-45b3-aafb-d50c3f3b9c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:01.042779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-eb9a845b-6061-45b3-aafb-d50c3f3b9c34 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:01.044080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-eb9a845b-6061-45b3-aafb-d50c3f3b9c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:01.065619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:01.146649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 677/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:01.426039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:01.427665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:01.533291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69df69-9c50-4307-9fdd-ec77cc8f8a15 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:01.608422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:01.609027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:01.609311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:01.609584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:01.611174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:01.656516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:01.724005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:01.725436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c08e1c-820d-422c-bcf3-b763d935560c service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:01.749957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 669/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:06.115467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:06.115949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:06.116042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:06.118023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 678/1347] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:06.123530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812b6a4-5188-4490-89bf-ccb7bc44225d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:06.124157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812b6a4-5188-4490-89bf-ccb7bc44225d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:06.124157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812b6a4-5188-4490-89bf-ccb7bc44225d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:06.190796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2812b6a4-5188-4490-89bf-ccb7bc44225d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:06.192076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2812b6a4-5188-4490-89bf-ccb7bc44225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:06.289642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:06.291872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 670/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:06.577096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:06.578651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:06.684644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb9922-1010-4a31-a7b5-eb26228a7edb None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:06.732822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:06.760455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:06.761245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:06.761697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:06.762244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:06.763899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:06.871314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:06.872699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add41f0-454d-4d96-85d5-705a286785ca service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:06.891676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 679/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:11.767278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec7c6-bf16-4765-9ec4-8d300d675984 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:11.767834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec7c6-bf16-4765-9ec4-8d300d675984 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:11.767834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec7c6-bf16-4765-9ec4-8d300d675984 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:11.770285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 671/1350] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:11.776768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda0f5a-64a0-457e-ad22-477d5d1e347d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:11.777204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda0f5a-64a0-457e-ad22-477d5d1e347d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:11.777526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda0f5a-64a0-457e-ad22-477d5d1e347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:11.878597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7eda0f5a-64a0-457e-ad22-477d5d1e347d None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:11.879990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7eda0f5a-64a0-457e-ad22-477d5d1e347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:11.919229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:12.078793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 680/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:19:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:16.106182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a859-c81a-4395-ab8c-6329f7cbeb18 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:16.106699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a859-c81a-4395-ab8c-6329f7cbeb18 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:16.106699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a859-c81a-4395-ab8c-6329f7cbeb18 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:16.108082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 672/1352] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:16.113316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe6d1-0c7d-4629-b51d-5ed6e9f06b1b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:16.113759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe6d1-0c7d-4629-b51d-5ed6e9f06b1b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:16.113862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe6d1-0c7d-4629-b51d-5ed6e9f06b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:16.203373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4a9fe6d1-0c7d-4629-b51d-5ed6e9f06b1b None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:16.204515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4a9fe6d1-0c7d-4629-b51d-5ed6e9f06b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:16.284061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:16.324300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 681/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:19:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:17.736237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b38c-1491-4148-8aa9-321ceadd291d None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:17.736778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b38c-1491-4148-8aa9-321ceadd291d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:17.736778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b38c-1491-4148-8aa9-321ceadd291d None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:17.738215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 673/1354] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:17.743516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae48f4-bd87-4073-94b2-74a1570361d8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:17.744001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae48f4-bd87-4073-94b2-74a1570361d8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:17.744245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae48f4-bd87-4073-94b2-74a1570361d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:17.831881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cdae48f4-bd87-4073-94b2-74a1570361d8 None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:17.833002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cdae48f4-bd87-4073-94b2-74a1570361d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:17.948651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 682/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:19:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:19:21.618254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871adc8d-3cf9-410e-8a70-aba9cfb95e6c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:21.618778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871adc8d-3cf9-410e-8a70-aba9cfb95e6c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:21.618778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871adc8d-3cf9-410e-8a70-aba9cfb95e6c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:21.620141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 674/1356] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:21.629686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4d7e-4b30-49cc-b836-a6de1f9e4640 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:21.630499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4d7e-4b30-49cc-b836-a6de1f9e4640 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:21.630792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e4d7e-4b30-49cc-b836-a6de1f9e4640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:21.668904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:21.701586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7c3e4d7e-4b30-49cc-b836-a6de1f9e4640 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:21.702776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7c3e4d7e-4b30-49cc-b836-a6de1f9e4640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:21.808781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 683/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:22.130517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:22.130755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:22.131031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:22.132382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 675/1358] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:22.140571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965252d-443f-4e28-a0b8-4cacbeabb254 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:22.140888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965252d-443f-4e28-a0b8-4cacbeabb254 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:22.141195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965252d-443f-4e28-a0b8-4cacbeabb254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:22.228432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e965252d-443f-4e28-a0b8-4cacbeabb254 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:22.229724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e965252d-443f-4e28-a0b8-4cacbeabb254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:22.294977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:22.367710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 684/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:22.375929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:22.377566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:22.534680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09596-488e-4c79-9206-b36d7062cb1e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:22.617013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:22.662013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:22.664481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:22.665173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:22.665811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:22.668230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:22.774171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:22.775835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7f5a7-87c3-4a20-8efa-e857e00107bc service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:22.799548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 676/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:25.192628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:25.193093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:25.193404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:25.195925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 685/1361] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:25.200505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff7a4-108a-4ef7-9821-e265a2c7c2ff None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:25.200906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff7a4-108a-4ef7-9821-e265a2c7c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:25.201084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff7a4-108a-4ef7-9821-e265a2c7c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:25.273627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c42ff7a4-108a-4ef7-9821-e265a2c7c2ff None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:25.274852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c42ff7a4-108a-4ef7-9821-e265a2c7c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:25.317629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:25.378731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 677/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:25.673987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:25.675498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:25.861560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56dea2-c5ce-4986-a3a5-b0e26f2e0aee None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:25.872215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:26.001637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:26.002251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:26.002534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:26.002807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:26.004612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:26.097141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:26.098749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca5ab-5d32-482c-9335-a10edfe5e365 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:26.118598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 686/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:28.731733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba613-ef9d-43e8-ad3d-a0531b2170ef None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:28.732424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba613-ef9d-43e8-ad3d-a0531b2170ef None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:28.732510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba613-ef9d-43e8-ad3d-a0531b2170ef None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:28.733962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 678/1364] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:28.739225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a1a8-e037-43f7-a137-fa4ad2909a00 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:28.739756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a1a8-e037-43f7-a137-fa4ad2909a00 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:28.739858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a1a8-e037-43f7-a137-fa4ad2909a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:28.755895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:28.811550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7ac7a1a8-e037-43f7-a137-fa4ad2909a00 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:28.812915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7ac7a1a8-e037-43f7-a137-fa4ad2909a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:28.919613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 687/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:29.215324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:29.215914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:29.216641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:29.219033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 679/1366] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:29.224545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefedff-ef17-4efb-8e26-dc4f45e1e1d2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:29.224868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefedff-ef17-4efb-8e26-dc4f45e1e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:29.233562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefedff-ef17-4efb-8e26-dc4f45e1e1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:29.308585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cdefedff-ef17-4efb-8e26-dc4f45e1e1d2 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:29.309838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cdefedff-ef17-4efb-8e26-dc4f45e1e1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:29.348302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:29.435195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 688/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:19:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:29.443119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:29.445452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:29.606842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6770-ffa5-455c-942e-9da948a51684 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:29.630645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:29.711423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a4e7-ba4e-4447-aa59-86227c001c90 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:29.711746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a4e7-ba4e-4447-aa59-86227c001c90 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:29.712029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a4e7-ba4e-4447-aa59-86227c001c90 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:29.713388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 689/1368] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:29.718304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94276c-903e-47bf-87ce-ef9ebe3a17d5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:29.718720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94276c-903e-47bf-87ce-ef9ebe3a17d5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:29.719099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94276c-903e-47bf-87ce-ef9ebe3a17d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:29.735195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:29.735834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:29.736117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:29.736393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:29.738007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:29.791113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-da94276c-903e-47bf-87ce-ef9ebe3a17d5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:29.792520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-da94276c-903e-47bf-87ce-ef9ebe3a17d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:29.836776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:29.838614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1f49a-bfbd-4405-b388-6f4ac5384bb4 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:29.862831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 680/1369] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:29.895633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:29.904898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 690/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:19:30.248516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:30.248516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:30.248516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:30.249539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 681/1371] 10.222.0.18 () {54 vars in 964 bytes} [Thu Sep 15 19:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:30.254474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5066ea-1bef-4db4-ae16-3fd5b1421e38 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:30.254768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5066ea-1bef-4db4-ae16-3fd5b1421e38 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:30.255051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5066ea-1bef-4db4-ae16-3fd5b1421e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:30.349444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1b5066ea-1bef-4db4-ae16-3fd5b1421e38 None None] MFA Rules not processed for user `2de7b02bd5864be1b8e8bc6e06dddd55`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:30.350798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1b5066ea-1bef-4db4-ae16-3fd5b1421e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:30.433953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 691/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Sep 15 19:19:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:30.442576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:30.444461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:30.477103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:30.556531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773b041-dfe8-45c7-9372-8811004530b3 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:30.637854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:30.638468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:30.638751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:30.639023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:30.640788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:30.845378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:30.885316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:30.886864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1c61a-e403-43f1-831a-b249addcd7a0 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:30.919916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 682/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:32.282439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:32.283003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:32.283003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:32.284373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 692/1374] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:32.290278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc505d-cda3-4062-85c0-f7d99a65369e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:32.290707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc505d-cda3-4062-85c0-f7d99a65369e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:32.290931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc505d-cda3-4062-85c0-f7d99a65369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:32.359635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f7cc505d-cda3-4062-85c0-f7d99a65369e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:32.360883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f7cc505d-cda3-4062-85c0-f7d99a65369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:32.456848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 683/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:32.749849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:32.751387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:32.842187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:32.907776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3bf93-2407-400d-964a-5cb00cfcedc4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:33.028719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:33.029340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:33.029618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:33.029892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:33.031493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:33.083354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:33.126332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:33.128057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81bc0c-2678-4018-9b7d-5228e2029205 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:33.147611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 693/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:35.490058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:35.490641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:35.490641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:35.492677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 684/1377] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:35.497696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b319794-be15-42c4-8d4a-356bb4df6f90 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:35.498118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b319794-be15-42c4-8d4a-356bb4df6f90 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:35.498254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b319794-be15-42c4-8d4a-356bb4df6f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:35.567463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0b319794-be15-42c4-8d4a-356bb4df6f90 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:35.568704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0b319794-be15-42c4-8d4a-356bb4df6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:35.674695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 694/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:35.963965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:35.965477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:36.008337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:36.130810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026c04-522f-4659-ad1a-ca20296e00e0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:36.246526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:36.251953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:36.257632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:36.257632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:36.257632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:36.257632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:36.349327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:36.350942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69a55c-a44a-4dd4-9b52-8faa4d7f8bea service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:36.371364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 685/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:38.812248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:38.812823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:38.812823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:38.814115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 695/1380] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:38.821351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f50b-fc76-42e5-9e80-9f4dbcb8768b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:38.821804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f50b-fc76-42e5-9e80-9f4dbcb8768b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:38.822105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f50b-fc76-42e5-9e80-9f4dbcb8768b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:38.909332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3799f50b-fc76-42e5-9e80-9f4dbcb8768b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:38.910613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3799f50b-fc76-42e5-9e80-9f4dbcb8768b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:38.978220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:39.011522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 686/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:39.309058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:39.310816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:39.344488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:39.474754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926f0d8-8103-46f3-ae35-fc5d74a9df34 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:39.577007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:39.594294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:39.594943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:39.595254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:39.595554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:39.597314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:39.692432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:39.694112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa80ad9-35f8-4c3a-8fb3-2f59cb29f981 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:39.713837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 696/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:42.313078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:42.313761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:42.313761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:42.315378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 687/1383] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:42.320987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434b3f-315b-4b0a-afb3-edbc19bd8448 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:42.321430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434b3f-315b-4b0a-afb3-edbc19bd8448 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:42.321600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434b3f-315b-4b0a-afb3-edbc19bd8448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:42.399190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3c434b3f-315b-4b0a-afb3-edbc19bd8448 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:42.400933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3c434b3f-315b-4b0a-afb3-edbc19bd8448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:42.466372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:42.509767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 697/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:42.812051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:42.814047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:42.973262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3dab0-f56c-45c7-b32c-274f917f5412 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:43.032707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:43.099768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:43.100283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:43.100566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:43.100836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:43.103004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:43.194267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:43.195897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4788e-5fe3-4e71-a846-bf5750fbb121 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:43.214767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 688/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:45.689714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:45.690289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:45.690289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:45.691777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 698/1386] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:45.697152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988a5a-2748-4124-9ad3-257dca33efa8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:45.697585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988a5a-2748-4124-9ad3-257dca33efa8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:45.697820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988a5a-2748-4124-9ad3-257dca33efa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:45.763636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-79988a5a-2748-4124-9ad3-257dca33efa8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:45.764869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79988a5a-2748-4124-9ad3-257dca33efa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:45.773940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:45.861911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 689/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:46.153351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:46.154873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:46.303415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb681ce-23d1-4fbb-ac66-bea46c7bd2db None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:46.354009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:46.422282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100a2be-650e-4cfe-9866-cfa1c11f9c0c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:46.422490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:46.422619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100a2be-650e-4cfe-9866-cfa1c11f9c0c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:46.422851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:46.423134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:46.423238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100a2be-650e-4cfe-9866-cfa1c11f9c0c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:46.423412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:46.424954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 690/1388] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:19:46.425335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:46.430101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a6eb-e9dc-4de4-bee2-64036b9b9dbb None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:46.430392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a6eb-e9dc-4de4-bee2-64036b9b9dbb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:46.430674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a6eb-e9dc-4de4-bee2-64036b9b9dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:46.497149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-abe6a6eb-e9dc-4de4-bee2-64036b9b9dbb None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:46.498523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-abe6a6eb-e9dc-4de4-bee2-64036b9b9dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:46.516566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:46.518163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb7ab8-2059-4a45-a911-2c59420afa6c service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:46.536778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 699/1389] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:46.587167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:46.598088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 691/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:19:46.897264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:46.897776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:46.897776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:46.899822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 700/1391] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:46.905082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de67d-c446-4b09-ab87-d2fcec17c7a9 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:46.905481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de67d-c446-4b09-ab87-d2fcec17c7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:46.905770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de67d-c446-4b09-ab87-d2fcec17c7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:46.984800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fa6de67d-c446-4b09-ab87-d2fcec17c7a9 None None] MFA Rules not processed for user `37e3dbce9a544a519e965dc35cd5ba9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:46.985920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fa6de67d-c446-4b09-ab87-d2fcec17c7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:47.063371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 692/1392] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:19:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:47.072925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:47.074407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:47.125511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:47.177177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc8e85-7830-4fe2-adfa-cf54ab4ac7f8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:47.253586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:47.254222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:47.254529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:47.254799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:47.256421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:47.355024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:47.367753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:47.369226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39ea55-2006-4a0f-b888-a6f8256e16db service glance] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:47.388531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 701/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:48.299148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a016c3-08aa-422a-a44f-dbcb3fa74ef1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.299618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a016c3-08aa-422a-a44f-dbcb3fa74ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.299777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a016c3-08aa-422a-a44f-dbcb3fa74ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.367273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-95a016c3-08aa-422a-a44f-dbcb3fa74ef1 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:48.368561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-95a016c3-08aa-422a-a44f-dbcb3fa74ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:48.404277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:48.456623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 693/1394] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:48.503261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.503745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.503881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.505277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 702/1395] 10.222.0.18 () {54 vars in 943 bytes} [Thu Sep 15 19:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:48.512165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317553e-483f-4b76-bca3-85f219c9a99b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.512457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317553e-483f-4b76-bca3-85f219c9a99b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.512737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317553e-483f-4b76-bca3-85f219c9a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.588273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d317553e-483f-4b76-bca3-85f219c9a99b None None] MFA Rules not processed for user `e0c4e352d9914c3a9a5225439274d91d`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:48.589468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d317553e-483f-4b76-bca3-85f219c9a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:48.707007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 694/1396] 10.222.0.18 () {58 vars in 1040 bytes} [Thu Sep 15 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:19:48.718499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:48.720032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:48.878407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67422-5f02-461f-bbd6-f88d2efc7b7a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:48.890515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec04c-2a87-46ee-9545-9de819fdeee3 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.890809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec04c-2a87-46ee-9545-9de819fdeee3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.891091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec04c-2a87-46ee-9545-9de819fdeee3 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.892871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 695/1397] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:19:48.898007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.898423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.898711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.908417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:48.952510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:48.975169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:48.976949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:48.992187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:48.994314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:48.994594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:48.994868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:48.996578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:49.076697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 696/1398] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:19:49.082765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:49.084412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27434c4-166a-47ea-a639-02eb73267a13 service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:49.093062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 703/1399] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:19:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:19:49.367560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:49.368809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:49.463861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:49.499606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea8d24-3342-4d3f-b4b6-3a463a14d272 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:49.631300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:49.631870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:49.632183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:49.632461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:49.634143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:49.719059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:49.730629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:49.733282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d5cbb-8c36-42b1-aa18-11f3f6dff398 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:49.754167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 697/1400] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:52.095994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:52.097598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:52.097890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:52.099278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 704/1401] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:52.105402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291cff-1681-4f31-ae10-b21e09ecb58f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:52.105833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291cff-1681-4f31-ae10-b21e09ecb58f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:52.106010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9291cff-1681-4f31-ae10-b21e09ecb58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:52.183044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a9291cff-1681-4f31-ae10-b21e09ecb58f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:52.184326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a9291cff-1681-4f31-ae10-b21e09ecb58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:52.297915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 698/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:52.589417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:52.591023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:52.719463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:52.779507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4b2f8-266c-4321-851d-1d1df17c3f44 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:52.922077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:52.923223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:52.923515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:52.924101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:52.931481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:53.011721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:53.036427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:53.038234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e09056-8811-498c-a9a6-71c575f4d409 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:53.062805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 705/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:55.463136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:55.463969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:55.463969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:55.468945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 699/1404] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:55.470602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c11b8-5812-4d17-894a-50f32e95db34 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:55.471810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c11b8-5812-4d17-894a-50f32e95db34 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:55.472202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c11b8-5812-4d17-894a-50f32e95db34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:55.549117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ab6c11b8-5812-4d17-894a-50f32e95db34 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:55.550678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ab6c11b8-5812-4d17-894a-50f32e95db34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:55.676297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 706/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:55.968917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:55.970391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:56.040760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:56.133627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2688a7-c4a8-4279-bb7f-40f946310432 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:56.263765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:56.264499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:56.264788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:56.265061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:56.266808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:56.290354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:56.376440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:56.378018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07a7ff-c331-4ddc-b271-df512cfd836b service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:56.400887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 700/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:19:58.752388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:58.752968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:58.752968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:58.754292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 707/1407] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:19:58.760216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e694-c1b5-48fb-a855-e61bdcd62960 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:58.760652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e694-c1b5-48fb-a855-e61bdcd62960 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:58.760751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e694-c1b5-48fb-a855-e61bdcd62960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:58.830972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-cd74e694-c1b5-48fb-a855-e61bdcd62960 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:19:58.830972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-cd74e694-c1b5-48fb-a855-e61bdcd62960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:58.938398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:58.944335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 701/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:19:59.232485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:19:59.234019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:59.284900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:59.401944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f25a3-0730-4fff-bc41-848a2602a365 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:19:59.576776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:19:59.577498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:19:59.577785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:19:59.578293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:19:59.580023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:19:59.583199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:19:59.677031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:19:59.678624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3c93c-4a12-4f0b-96ef-01d2db658562 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:19:59.697873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 708/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:08.982648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:09.696137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=43 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:13.219641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a4ad1-5249-4638-bfb5-c6e3f1fc410f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:13.220199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a4ad1-5249-4638-bfb5-c6e3f1fc410f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:13.220199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a4ad1-5249-4638-bfb5-c6e3f1fc410f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:13.221555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 702/1410] 10.222.0.18 () {54 vars in 918 bytes} [Thu Sep 15 19:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:13.226524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1fc3-fbec-4e57-b478-9a455bfb4cc5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:13.226934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1fc3-fbec-4e57-b478-9a455bfb4cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:13.227077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1fc3-fbec-4e57-b478-9a455bfb4cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:13.303339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b83c1fc3-fbec-4e57-b478-9a455bfb4cc5 None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:13.304576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b83c1fc3-fbec-4e57-b478-9a455bfb4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:13.413305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 709/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 15 19:20:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:13.428841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2155e8-c8cb-4c2c-9670-f40ca8684204 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:13.429109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2155e8-c8cb-4c2c-9670-f40ca8684204 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:13.429387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2155e8-c8cb-4c2c-9670-f40ca8684204 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:13.430743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 703/1412] 10.222.0.18 () {54 vars in 918 bytes} [Thu Sep 15 19:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:13.435783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41490-fdbe-4bbe-855a-f713ab9e3807 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:13.436071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41490-fdbe-4bbe-855a-f713ab9e3807 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:13.436343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41490-fdbe-4bbe-855a-f713ab9e3807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:13.510427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:13.513439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-80c41490-fdbe-4bbe-855a-f713ab9e3807 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:13.518539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-80c41490-fdbe-4bbe-855a-f713ab9e3807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:13.630424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 710/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 15 19:20:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:20:16.202288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:16.202808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:16.202808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:16.204162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:20:16.209579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b909ad-6ab4-40c8-911f-8128fb547ef6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:16.209998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b909ad-6ab4-40c8-911f-8128fb547ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:16.210141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b909ad-6ab4-40c8-911f-8128fb547ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:16.277515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d4b909ad-6ab4-40c8-911f-8128fb547ef6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:16.278719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d4b909ad-6ab4-40c8-911f-8128fb547ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:16.333540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:16.377789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:16.678073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:16.679576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:16.825933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08034fa-50d5-4165-b69d-a01d5dd8dc8e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:16.898587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:16.939416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:16.940052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:16.940354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:16.940614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:16.942213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:17.033564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:17.035146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44562063-e166-428d-b513-95b5ebc34559 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:17.053261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 705/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:18.191422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:18.191969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:18.191969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:18.193323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:18.199678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f482-ad8a-40c3-bfd2-fe46c80ba01b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:18.200234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f482-ad8a-40c3-bfd2-fe46c80ba01b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:18.200234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f482-ad8a-40c3-bfd2-fe46c80ba01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:18.275428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9484f482-ad8a-40c3-bfd2-fe46c80ba01b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:18.276670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9484f482-ad8a-40c3-bfd2-fe46c80ba01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:18.303376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:18.371906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:18.522812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:18.524965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:18.670395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327913a-3058-4ebb-8d5d-4fa47cb4ef56 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:18.709742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:18.782066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:18.782576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:18.782856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:18.783135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:18.784766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:18.872187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:18.873773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90a6b-0fb5-4818-b59a-35e6a06b50d6 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:18.891774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 713/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:20.021543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:20.023668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:20.023668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:20.023668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:20.030439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4846d2-49be-4c22-aa8d-a5db36eac445 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:20.030910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4846d2-49be-4c22-aa8d-a5db36eac445 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:20.031083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4846d2-49be-4c22-aa8d-a5db36eac445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:20.084801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:20.107578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dd4846d2-49be-4c22-aa8d-a5db36eac445 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:20.108797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dd4846d2-49be-4c22-aa8d-a5db36eac445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:20.204404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:20.359633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:20.361210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:20.505930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ffd0-cc88-4961-904d-6c16dfabd847 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:20.518168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:20.617021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:20.618124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:20.618404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:20.618675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:20.620436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:20.708343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:20.709908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0b4aa-c6a7-4d12-9f69-a6eedf9ab94b service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:20.728398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 708/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:21.835708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:21.836252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:21.836252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:21.837674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:21.844285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3802397-bcf5-42a4-afa4-8d4df389f865 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:21.844707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3802397-bcf5-42a4-afa4-8d4df389f865 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:21.844834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3802397-bcf5-42a4-afa4-8d4df389f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:21.864723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:21.922204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d3802397-bcf5-42a4-afa4-8d4df389f865 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:21.923430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d3802397-bcf5-42a4-afa4-8d4df389f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:22.019687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:22.170146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:22.171608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:22.283508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:22.318865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5d723-a9b8-4d91-b722-8f49a607847d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:22.431781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:22.432299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:22.432579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:22.432851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:22.434447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:22.510435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:22.523679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:22.525857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af4cc9-10bc-4e52-a0bb-5374c957bf45 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:22.543762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 716/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:23.664981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:23.665540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:23.665540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:23.666985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:23.673170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789334-dbef-42e0-8311-611bc7343e5d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:23.673616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789334-dbef-42e0-8311-611bc7343e5d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:23.673740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789334-dbef-42e0-8311-611bc7343e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:23.748823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-fa789334-dbef-42e0-8311-611bc7343e5d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:23.750043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fa789334-dbef-42e0-8311-611bc7343e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:23.850955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:23.999625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:24.001218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:24.085336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:24.147091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0131-1e28-49fc-98c9-7c0cea86d9e5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:24.257525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:24.258674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:24.258904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:24.259181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:24.260982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:24.311378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:24.354154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:24.354154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b4d6e-44de-4a4d-90b3-d9519570ba49 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:24.372268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 711/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:25.493956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:25.494500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:25.494500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:25.495900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:25.502315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555df1-be4a-488e-8b60-10d8b2b6f935 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:25.502738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555df1-be4a-488e-8b60-10d8b2b6f935 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:25.502877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555df1-be4a-488e-8b60-10d8b2b6f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:25.587119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ac555df1-be4a-488e-8b60-10d8b2b6f935 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:25.588424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ac555df1-be4a-488e-8b60-10d8b2b6f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:25.682940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:25.833980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:25.835448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:25.886835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:25.982746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26f444-de78-4534-8d55-df7905c1fe59 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:26.095289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:26.095842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:26.096129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:26.096408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:26.098009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:26.105037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:26.188480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:26.190127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a7b09-09a9-409f-9eec-8a4db13aa892 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:26.207987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 719/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:27.338233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:27.338810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:27.338810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:27.340169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:27.346443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a7e3-4663-432c-bdcb-9c76fe5f794f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:27.346891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a7e3-4663-432c-bdcb-9c76fe5f794f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:27.347039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a7e3-4663-432c-bdcb-9c76fe5f794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:27.423321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4b51a7e3-4663-432c-bdcb-9c76fe5f794f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:27.424803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4b51a7e3-4663-432c-bdcb-9c76fe5f794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:27.491228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:27.520261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:27.670365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:27.671893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:27.689489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:27.816518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d5454-bb89-4763-99fc-62fff807a827 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:27.905521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:27.930652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:27.931264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:27.931546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:27.931852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:27.938102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:28.022040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:28.023616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0bff3-1b3e-45fa-936f-dacd72ee1651 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:28.042049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 714/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:29.183626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:29.184188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:29.184188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:29.185531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:29.191850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be54d9-a7c8-4ca0-8d68-54a19f08ae73 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:29.192259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be54d9-a7c8-4ca0-8d68-54a19f08ae73 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:29.192407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be54d9-a7c8-4ca0-8d68-54a19f08ae73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:29.267664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-84be54d9-a7c8-4ca0-8d68-54a19f08ae73 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:29.268884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-84be54d9-a7c8-4ca0-8d68-54a19f08ae73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:29.310144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:29.363769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:29.516246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:29.517761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:29.660371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa957f-28c0-4ea1-b86f-14c116313248 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:29.713649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:29.771781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:29.772313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:29.772595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:29.772869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:29.774561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:29.864055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:29.865636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241a3a6-8d7e-4ca8-a08e-b133b9945f97 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:29.884237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 722/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:30.984289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:30.984807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:30.984905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:30.986219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:30.992580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5753db-be80-4f00-bbda-616f95ac2d27 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:30.992996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5753db-be80-4f00-bbda-616f95ac2d27 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:30.993136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5753db-be80-4f00-bbda-616f95ac2d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:31.068720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3e5753db-be80-4f00-bbda-616f95ac2d27 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:31.069929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3e5753db-be80-4f00-bbda-616f95ac2d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:31.072472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:31.167904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:31.318001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:31.319567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:31.464137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e815f-b737-4522-8bfc-89ea22e8769f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:31.490536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:31.574983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:31.575484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:31.575805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:31.576083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:31.577668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:31.664120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:31.666205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56da49-3613-4895-ae9d-dfda9a42b748 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:31.684469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 717/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:33.763673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:33.764228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:33.764346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:33.765610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:33.770780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3695-da47-4bb6-a082-7dc3f30e4a22 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:33.771199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3695-da47-4bb6-a082-7dc3f30e4a22 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:33.771332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3695-da47-4bb6-a082-7dc3f30e4a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:33.812526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:33.837940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7f7f3695-da47-4bb6-a082-7dc3f30e4a22 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:33.839278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7f7f3695-da47-4bb6-a082-7dc3f30e4a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:33.934785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:34.220850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:34.222396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:34.326113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323350f9-8da8-40c3-8e01-dff1166d4ca3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:34.359254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:34.400786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:34.401393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:34.401694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:34.401950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:34.403559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:34.511353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:34.512829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0df2-ba38-40e2-b928-114e0b1dc9d7 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:34.530978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 725/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:36.906955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:36.907524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:36.907524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:36.908872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:36.914188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e110-b904-405e-b2e1-596dfe38a55d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:36.914608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e110-b904-405e-b2e1-596dfe38a55d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:36.914732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e110-b904-405e-b2e1-596dfe38a55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:36.981142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ce76e110-b904-405e-b2e1-596dfe38a55d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:36.982349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ce76e110-b904-405e-b2e1-596dfe38a55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:36.995142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:37.082404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:37.370051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:37.371578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:37.472392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b69677-aa54-49a6-a265-b8f6244cc17f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:37.485645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:37.548615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:37.549209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:37.549488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:37.550024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:37.551595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:37.657433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:37.658787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376ee1-7261-4386-b3df-cbec363307f3 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:37.676730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 720/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:40.191733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:40.192247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:40.192247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:40.193540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:40.198905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419d0-6293-4050-9b41-28cfae9f817a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:40.199328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419d0-6293-4050-9b41-28cfae9f817a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:40.199470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419d0-6293-4050-9b41-28cfae9f817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:40.247376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:40.264558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-903419d0-6293-4050-9b41-28cfae9f817a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:40.266905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-903419d0-6293-4050-9b41-28cfae9f817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:40.360053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:40.646918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:40.649127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:40.749454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e9af9-f428-4751-9d10-66ecc57274c5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:40.789597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:40.822749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:40.823351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:40.823633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:40.823930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:40.825525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:40.940120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:40.941587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708fa17-f48c-4c58-95d6-75855ad68605 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:40.959705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 728/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:43.275779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:43.276309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:43.276522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:43.277682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:43.283172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342ab3e-f768-4010-8f1a-f5da42183b46 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:43.283617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342ab3e-f768-4010-8f1a-f5da42183b46 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:43.283760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342ab3e-f768-4010-8f1a-f5da42183b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:43.349808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a342ab3e-f768-4010-8f1a-f5da42183b46 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:43.351039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a342ab3e-f768-4010-8f1a-f5da42183b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:43.371725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:43.448502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:43.730335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:43.731912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:43.835686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7f625-6c66-45e9-a1a5-d59f267e1706 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:43.856967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:43.910931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:43.911546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:43.911860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:43.912136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:43.913773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:44.023310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:44.025307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70754236-c04c-48f9-af23-3f3c373b140e service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:44.043475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 723/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:46.242204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:46.242724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:46.242839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:46.244191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:46.249391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663734d-3a34-472a-8e15-5f71b785e63e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:46.249832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663734d-3a34-472a-8e15-5f71b785e63e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:46.249955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663734d-3a34-472a-8e15-5f71b785e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:46.312982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:46.315778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1663734d-3a34-472a-8e15-5f71b785e63e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:46.316981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1663734d-3a34-472a-8e15-5f71b785e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:46.413103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:46.697989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:46.699535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:46.802418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809b4e1-16f5-45d5-844f-ec05d076b55e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:46.851923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:46.875822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:46.876424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:46.876705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:46.876973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:46.878550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:46.989468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:46.990956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438a043-c1df-4e84-bf1b-f7853570018c service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:47.010475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 731/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:49.259288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:49.259840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:49.259970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:49.261232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:20:49.266603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb76-7ae2-4dfc-938d-a9ad2e71832f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:49.267007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb76-7ae2-4dfc-938d-a9ad2e71832f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:49.267210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb76-7ae2-4dfc-938d-a9ad2e71832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:49.332524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0c46bb76-7ae2-4dfc-938d-a9ad2e71832f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:49.333719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0c46bb76-7ae2-4dfc-938d-a9ad2e71832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:49.362966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:49.429663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:49.719581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:49.721157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:49.823483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ae3c2-ee22-4bba-b890-772d3313eebd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:49.852645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:49.898867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:49.899465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:49.899771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:49.900044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:49.901634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:50.012576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:50.014008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52edcf-0b5d-4a45-ab04-8c03b8049e37 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:50.031808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 726/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:52.253023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:52.253581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:52.253581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:52.254925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:52.260269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571d54f-872b-4035-aaae-5bf856438cac None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:52.260695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571d54f-872b-4035-aaae-5bf856438cac None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:52.260864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571d54f-872b-4035-aaae-5bf856438cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:52.326800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6571d54f-872b-4035-aaae-5bf856438cac None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:52.329090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6571d54f-872b-4035-aaae-5bf856438cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:52.339027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:52.427194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:52.708787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:52.710471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:52.817720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e51e-45bc-4fd0-9427-11bb05c51973 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:52.873961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:52.892530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:52.893134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:52.893416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:52.893689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:52.895280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:53.004040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:53.005490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fd408-b873-4d41-9b9b-e7d47cc70afc service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:53.024319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 734/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:55.219633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:55.220226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:55.220226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:55.221506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:55.226829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e45b-18a8-4238-9bb5-a066a5bc208a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:55.227220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e45b-18a8-4238-9bb5-a066a5bc208a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:55.227386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e45b-18a8-4238-9bb5-a066a5bc208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:55.296269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9754e45b-18a8-4238-9bb5-a066a5bc208a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:55.297644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9754e45b-18a8-4238-9bb5-a066a5bc208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:55.337030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:55.394745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:55.676639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:55.678174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:55.781439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54106091-145b-4a21-a763-5fb2ce615f77 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:55.818751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:55.855234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:55.855996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:55.856284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:55.856559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:55.858190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:55.969081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:55.970589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5e474-c77d-4476-a1f6-32870d47aac5 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:55.989468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 729/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:20:58.197668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:58.198240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:58.198240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:58.199485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:20:58.204951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8a28-f308-43d5-b548-256a1ec554e4 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:58.205380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8a28-f308-43d5-b548-256a1ec554e4 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:58.205521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8a28-f308-43d5-b548-256a1ec554e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:58.270588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fd1b8a28-f308-43d5-b548-256a1ec554e4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:20:58.271821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fd1b8a28-f308-43d5-b548-256a1ec554e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:58.291229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:58.369149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:20:58.657659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:20:58.659192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:58.763365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0fca4-f56b-4f46-80b8-114d5b4acdb3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:20:58.830893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:20:58.837961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:20:58.838616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:20:58.838902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:20:58.839175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:20:58.840832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:20:58.950120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:20:58.951553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05b195-b8d5-4079-870b-cb36b04ab216 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:20:58.969286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 737/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:01.152340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:01.152928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:01.152928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:01.154256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:01.159715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6534-af72-4171-a001-9d5de77fbcc7 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:01.160200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6534-af72-4171-a001-9d5de77fbcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:01.160304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6534-af72-4171-a001-9d5de77fbcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:01.225895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-678d6534-af72-4171-a001-9d5de77fbcc7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:01.227626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-678d6534-af72-4171-a001-9d5de77fbcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:01.276662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:01.324260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:01.613711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:01.615269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:01.717177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df95b6-ffd4-4520-8897-2544baaa4a49 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:01.764098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:01.789808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:01.790411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:01.790688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:01.790961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:01.792580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:01.901799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:01.903195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992ebb8-8853-49c4-9e76-2f5d25634999 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:01.921831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 732/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:11.337598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=22 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:11.919693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=66 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:21.760359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455eb16-983e-4439-b890-e6e86d5345f8 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:21.760969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455eb16-983e-4439-b890-e6e86d5345f8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:21.761324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455eb16-983e-4439-b890-e6e86d5345f8 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:21.763806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 739/1471] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:21.769083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c08f-b174-453d-94cc-b8636b7eda53 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:21.769526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c08f-b174-453d-94cc-b8636b7eda53 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:21.769632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c08f-b174-453d-94cc-b8636b7eda53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:21.836396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9b85c08f-b174-453d-94cc-b8636b7eda53 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:21.837640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9b85c08f-b174-453d-94cc-b8636b7eda53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:21.940512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 733/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:22.236158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:22.236442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:22.236713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:22.238039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 740/1473] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:22.243813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0e1f-f5f4-4398-8e0c-88eefcd3ffd8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:22.244107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0e1f-f5f4-4398-8e0c-88eefcd3ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:22.244387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0e1f-f5f4-4398-8e0c-88eefcd3ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:22.320313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-69eb0e1f-f5f4-4398-8e0c-88eefcd3ffd8 None None] MFA Rules not processed for user `37e3dbce9a544a519e965dc35cd5ba9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:22.321510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-69eb0e1f-f5f4-4398-8e0c-88eefcd3ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:22.341971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:22.397390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 734/1474] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:21:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:22.405183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:22.406688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:22.508587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e42cd-754d-4349-aa35-3137c485d6e8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:22.699779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:22.700390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:22.700676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:22.700954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:22.702591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:22.757995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:22.813833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:22.815305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a36b3-ae32-4aac-b2c0-8de475a87c47 service glance] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:22.833449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 741/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:25.068015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:25.068579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:25.068579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:25.070486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 735/1476] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:25.075762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0259aa-ff82-424a-b99c-1208f1b1629c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:25.076165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0259aa-ff82-424a-b99c-1208f1b1629c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:25.076323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0259aa-ff82-424a-b99c-1208f1b1629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:25.141418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6d0259aa-ff82-424a-b99c-1208f1b1629c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:25.142607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6d0259aa-ff82-424a-b99c-1208f1b1629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:25.237684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 742/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:25.510483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:25.511971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:25.635635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321897f-6950-47dc-a44a-3c88c4f46584 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:25.684183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:25.728642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:25.729245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:25.729526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:25.729801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:25.730443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:25.732187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:21:25.732187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:21:25.732187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:21:25.733327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fd7c0-d526-4829-b441-91d4d1e83e3a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:25.752008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 736/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 15 19:21:25 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:21:27.932848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:27.933265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:27.933405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:27.934755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 743/1479] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:27.940068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c10d-1d30-4455-8598-b4d168f7e6ae None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:27.940504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c10d-1d30-4455-8598-b4d168f7e6ae None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:27.940699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c10d-1d30-4455-8598-b4d168f7e6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:28.009003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6e24c10d-1d30-4455-8598-b4d168f7e6ae None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:28.010198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6e24c10d-1d30-4455-8598-b4d168f7e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:28.104340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 737/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:28.413040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:28.415157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:28.425942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:28.562148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1095c23-2887-4d60-820b-b2322cb1fc3e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:28.642733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:28.674967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:28.675878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:28.676370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:28.676805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:28.678735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:28.766605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:28.768235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1ae96-8f60-4813-89b2-71933cc61068 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:28.786496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 744/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:31.103340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:31.103933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:31.103933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:31.105284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 738/1482] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:31.110721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408791b0-d3aa-4141-8941-34a7edd84214 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:31.111119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408791b0-d3aa-4141-8941-34a7edd84214 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:31.111283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408791b0-d3aa-4141-8941-34a7edd84214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:31.178069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-408791b0-d3aa-4141-8941-34a7edd84214 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:31.179498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-408791b0-d3aa-4141-8941-34a7edd84214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:31.223980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:31.273575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 745/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:31.559989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:31.561515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:31.585989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:31.666249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66f0d-99b7-4ed4-8132-c9c694c8e48e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:31.738838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:31.739458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:31.739776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:31.740063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:31.741686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:31.815042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:31.849853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:31.851288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e1a0-24d6-4224-8f3b-087e7ac98908 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:31.869555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 739/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:34.065353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:34.065911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:34.065911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:34.067250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 746/1485] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:34.072614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4b4-3627-40d7-a01a-1c20aaf9c367 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:34.073048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4b4-3627-40d7-a01a-1c20aaf9c367 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:34.073196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4b4-3627-40d7-a01a-1c20aaf9c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:34.138194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9ed5b4b4-3627-40d7-a01a-1c20aaf9c367 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:34.139404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ed5b4b4-3627-40d7-a01a-1c20aaf9c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:34.233822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 740/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:21:34.520497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:34.522110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:34.623742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654962ec-5ccf-45fc-8cbb-302f9f7133bd None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:34.696202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:34.697648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:34.698244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:34.698526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:34.698800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:34.700437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:34.806944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:34.808362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f84c-8627-4ac9-9289-651106a2f38e service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:34.826129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 747/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:37.120946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:37.121500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:37.121500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:37.122844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 741/1488] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:37.128258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c1203-9344-459d-9fd3-7055b195c322 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:37.128660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c1203-9344-459d-9fd3-7055b195c322 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:37.128806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c1203-9344-459d-9fd3-7055b195c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:37.193867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-794c1203-9344-459d-9fd3-7055b195c322 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:37.195595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-794c1203-9344-459d-9fd3-7055b195c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:37.246884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:37.304435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 748/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:37.595323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:37.596896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:37.632902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:37.700008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331491-0b42-4de6-9773-d90885d0b424 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:37.773796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:37.774405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:37.774690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:37.774964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:37.776600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:37.858277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:37.887528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:37.888985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3393-75e2-4852-b683-4482564f74ec service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:37.907660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 742/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:40.442364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:40.442852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:40.442958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:40.444312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 749/1491] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 15 19:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:21:40.449660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e6625-56e8-4b6d-b3fc-b19a165a8004 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:40.450069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e6625-56e8-4b6d-b3fc-b19a165a8004 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:40.450209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e6625-56e8-4b6d-b3fc-b19a165a8004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:40.514820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-786e6625-56e8-4b6d-b3fc-b19a165a8004 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:21:40.516043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-786e6625-56e8-4b6d-b3fc-b19a165a8004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:40.611355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 743/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 15 19:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:21:40.902607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:21:40.904166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:41.008509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f5632-758a-4bd6-9f1b-2b7116b929f1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:21:41.082430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:21:41.083034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:21:41.083313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:21:41.083582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:21:41.085347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:21:41.094003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:41.195335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:21:41.196708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a814a-7b2f-4ccb-b387-6a5a2bb5fb5e service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:21:41.214525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 750/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:21:50.609594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=82 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:21:51.212371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=50 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:02.188378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:02.188931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:02.188931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:02.264966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:23:02.266174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:02.361952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 744/1494] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:23:02.370027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:02.370507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:02.370703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:02.444712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:23:02.445908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:02.540708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 751/1495] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:23:02.545328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:02.546537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:02.558108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7c8dc-7a61-4df3-8250-562c2ce50ca0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:02.618756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:02.651535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:02.652178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:02.652659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:02.653267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:02.653933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:23:02.654276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:02.655316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:02.655316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:23:02.655316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:23:02.656237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:02.667382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 745/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:23:02.675642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:02.685374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f45e3-5df1-49d9-b799-d6711161d0ed None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:02.747500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:02.781233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:02.781891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:02.782174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:02.782442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:02.783059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:02.784800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:02.784800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:23:02.784800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:23:02.786030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:02.893155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 752/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:23:02 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:23:02.897481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:02.899037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:02.933647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee86cf-f2eb-4c6c-a931-441599566215 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:03.026319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:03.026926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.027234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.027503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.028159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:03.029167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:03.029167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:23:03.029167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:23:03.030419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:03.116868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 746/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:23:02 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:23:03.120991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:03.122549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:03.183915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5160-3ad0-4dc6-bc90-b7d9e48665aa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:03.265283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:03.275220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:03.275852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.276160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.276485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.277080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:03.278105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:03.278105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:23:03.278105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:23:03.279013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:03.292879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 753/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:23:03.301894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:03.304066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:03.315338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39f480-9336-4f42-954b-3a34b77f6f83 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:03.340100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:03.407614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:03.408306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.408584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.408855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.409512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:03.410519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:03.410519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:23:03.410519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:23:03.411392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180db313-1b1b-4740-9f6e-fd177c789660 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:03.425710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 747/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:23:03.430245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:03.431845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:03.464911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f6265-23aa-4578-91e9-7368a1bbd8e2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:03.558841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:03.559432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.559735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.560010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] PATH_INFO: `/v3/projects/07ce66c4834d41a9aa93181ea7d22ca0/users/eeac30a83bef4a2dbfc356fdc149f59c/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.582512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ce66c4834d41a9aa93181ea7d22ca0, user_id=eeac30a83bef4a2dbfc356fdc149f59c, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:03.583532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:03.583532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:23:03.583532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:23:03.585510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb54a0-5300-4d28-9bda-1e74ae8a2828 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:03.714034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 754/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:23:03 2022] PUT /identity/v3/projects/07ce66c4834d41a9aa93181ea7d22ca0/users/eeac30a83bef4a2dbfc356fdc149f59c/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:23:03.726639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439841a-a884-4998-9590-d540d7bc2cf6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.726933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439841a-a884-4998-9590-d540d7bc2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.727213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439841a-a884-4998-9590-d540d7bc2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.782834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b439841a-a884-4998-9590-d540d7bc2cf6 None None] MFA Rules not processed for user `eeac30a83bef4a2dbfc356fdc149f59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:23:03.784085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b439841a-a884-4998-9590-d540d7bc2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:03.861715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 748/1502] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:23:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:23:03.875548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.875863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.876141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.877469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 755/1503] 10.222.0.18 () {54 vars in 964 bytes} [Thu Sep 15 19:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:23:03.882448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00fd3a-de68-4ec1-895c-878733e22985 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:03.882740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00fd3a-de68-4ec1-895c-878733e22985 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:03.883017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00fd3a-de68-4ec1-895c-878733e22985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:03.914637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:03.959743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6a00fd3a-de68-4ec1-895c-878733e22985 None None] MFA Rules not processed for user `2de7b02bd5864be1b8e8bc6e06dddd55`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:23:03.960945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6a00fd3a-de68-4ec1-895c-878733e22985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:04.034435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 749/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Sep 15 19:23:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:23:04.042741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:04.044281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:04.094026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:04.154318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979e313-b814-4570-a363-2df9d60aca51 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:04.229438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:04.230040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:04.230317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:04.230615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:04.232236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:04.328400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:04.329818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:04.333210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:04.347863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 756/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:23:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:23:04.393342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2fca-f8ec-4ee2-85dc-a4521f01ffc2 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:04.393779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2fca-f8ec-4ee2-85dc-a4521f01ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:04.393892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2fca-f8ec-4ee2-85dc-a4521f01ffc2 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:04.395140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 750/1506] 10.222.0.18 () {52 vars in 830 bytes} [Thu Sep 15 19:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:23:04.399294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:04.400547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:04.497953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d174-cb96-4398-a079-cf8c979670c2 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:04.589952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:04.590569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:04.590844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:04.591114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] PATH_INFO: `/v3/users/eeac30a83bef4a2dbfc356fdc149f59c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:04.603780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeac30a83bef4a2dbfc356fdc149f59c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:04.604791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:04.604791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:23:04.604791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:23:04.606005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:04.615440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:04.803046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeac30a83bef4a2dbfc356fdc149f59c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:04.879740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeac30a83bef4a2dbfc356fdc149f59c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:04.880272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeac30a83bef4a2dbfc356fdc149f59c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:04.880702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeac30a83bef4a2dbfc356fdc149f59c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:04.898492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeac30a83bef4a2dbfc356fdc149f59c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:04.927403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 757/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:23:04 2022] DELETE /identity/v3/users/eeac30a83bef4a2dbfc356fdc149f59c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:23:04.932323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:04.932612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:04.932888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:04.989106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:23:04.990820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:05.065160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:05.077841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 751/1508] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:23:05.088725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:05.095029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:05.188730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0d17b-41e8-4636-9ffb-7550fdb3eff8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:05.242353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:05.261135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:05.261735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:05.262015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:05.262286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:05.263894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:05.372270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:05.373635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1549a3-846e-4dea-ac6e-22c7735beec5 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:05.392054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 758/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:23:05.864889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:23:05.866122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:23:05.877669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4ed9e-4a01-4a71-a742-08b924e27a4e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:23:05.980671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:23:05.981329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:23:05.981609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:23:05.981880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] PATH_INFO: `/v3/projects/07ce66c4834d41a9aa93181ea7d22ca0` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:23:05.991500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ce66c4834d41a9aa93181ea7d22ca0)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:23:05.992544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:23:05.992544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:23:05.992544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:23:05.993722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:23:06.102051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ce66c4834d41a9aa93181ea7d22ca0', 'request_id': 'req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:06.115101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ce66c4834d41a9aa93181ea7d22ca0', 'request_id': 'req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:06.165154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ce66c4834d41a9aa93181ea7d22ca0', 'request_id': 'req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:06.188314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] The token cache is being invalidate because project 07ce66c4834d41a9aa93181ea7d22ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:23:06.188540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:06.188967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1f49ae7-cc1f-4668-894a-e4bfdf883f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:23:06.190357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 752/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:23:05 2022] DELETE /identity/v3/projects/07ce66c4834d41a9aa93181ea7d22ca0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:23:15.389776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=62 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:23:16.186153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:24:35.088613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:35.089216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:35.089216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:35.090512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:35.095445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d01c-f9e9-47f7-8362-2d3d934846c5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:35.095911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d01c-f9e9-47f7-8362-2d3d934846c5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:35.096123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d01c-f9e9-47f7-8362-2d3d934846c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:35.182133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d3b2d01c-f9e9-47f7-8362-2d3d934846c5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:35.183255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d3b2d01c-f9e9-47f7-8362-2d3d934846c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:35.285854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:35.439520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:35.441664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:35.563548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c249cc-66f6-4182-a1dc-c8d25a2e3c6d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:35.569619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:35.570229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:35.570510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:35.570780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:35.579425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:35.581095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9c44e-42e5-4db3-87b4-22bfc71ceecf service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:35.591380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 760/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:36.691059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9730-dfca-45ae-934e-59a582c94b34 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:36.691620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9730-dfca-45ae-934e-59a582c94b34 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:36.691620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9730-dfca-45ae-934e-59a582c94b34 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:36.693015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:36.699825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afffef-1b21-4444-9d47-8df4388ce522 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:36.700355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afffef-1b21-4444-9d47-8df4388ce522 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:36.700355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afffef-1b21-4444-9d47-8df4388ce522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:36.721465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-66afffef-1b21-4444-9d47-8df4388ce522 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:36.722693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-66afffef-1b21-4444-9d47-8df4388ce522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:36.732218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:36.886746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c9730-dfca-45ae-934e-59a582c94b34 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:36.902247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c9730-dfca-45ae-934e-59a582c94b34 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:36.908520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:36.909124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:36.909412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:36.909691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:36.918118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:36.919718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c7930-a20c-4712-b6d8-01822d610bd0 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:36.929565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 755/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:37.228712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4481-7ad7-422b-9dce-0092998c925e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:37.229058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4481-7ad7-422b-9dce-0092998c925e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:37.229541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4481-7ad7-422b-9dce-0092998c925e None None] PATH_INFO: `/v3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:37.230973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 762/1517] 10.222.0.33 () {54 vars in 925 bytes} [Thu Sep 15 19:24:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:24:37.237304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7227-473d-4df4-80fd-c02d5132d979 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:37.237595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7227-473d-4df4-80fd-c02d5132d979 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:37.237872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7227-473d-4df4-80fd-c02d5132d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:37.304339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-5b5a7227-473d-4df4-80fd-c02d5132d979 None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:37.305557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5b5a7227-473d-4df4-80fd-c02d5132d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:37.324364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:24:37.380632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 756/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 15 19:24:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:41.181065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713a95-a1c0-4076-afa3-589d35aba80f None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:41.181566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713a95-a1c0-4076-afa3-589d35aba80f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:41.181566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713a95-a1c0-4076-afa3-589d35aba80f None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:41.182962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 763/1519] 10.222.0.18 () {54 vars in 961 bytes} [Thu Sep 15 19:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:41.188075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e66d6-528c-4837-b07f-ee154321b1df None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:41.188520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e66d6-528c-4837-b07f-ee154321b1df None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:41.188640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e66d6-528c-4837-b07f-ee154321b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:41.209656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7e5e66d6-528c-4837-b07f-ee154321b1df None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:41.210851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7e5e66d6-528c-4837-b07f-ee154321b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:41.219556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 757/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Sep 15 19:24:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:41.227564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe713a95-a1c0-4076-afa3-589d35aba80f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:41.242293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe713a95-a1c0-4076-afa3-589d35aba80f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:41.247211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:41.247831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:41.248111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:41.248380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:41.256479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:41.258126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdd977-3cee-4257-b0d5-7fa9fef9c669 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:41.267425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 764/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:48.399419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e259e0-46f3-4d42-973a-7c8c322cd6c1 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:48.400019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e259e0-46f3-4d42-973a-7c8c322cd6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:48.400019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e259e0-46f3-4d42-973a-7c8c322cd6c1 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:48.401352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:48.407473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434986d-4e0f-4610-b8a5-6abe4525a2c4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:48.407952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434986d-4e0f-4610-b8a5-6abe4525a2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:48.408058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434986d-4e0f-4610-b8a5-6abe4525a2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:48.427124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8434986d-4e0f-4610-b8a5-6abe4525a2c4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:48.429214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8434986d-4e0f-4610-b8a5-6abe4525a2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:48.437286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:48.590321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e259e0-46f3-4d42-973a-7c8c322cd6c1 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:48.604507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e259e0-46f3-4d42-973a-7c8c322cd6c1 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:48.609840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:48.610433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:48.610718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:48.610997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:48.619360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:48.620984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dce999-2ad2-48d8-b56b-27f39f0f49c2 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:48.630286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 759/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:49.698442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6de96-e63b-4770-95b6-665508134d86 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:49.699017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6de96-e63b-4770-95b6-665508134d86 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:49.699017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6de96-e63b-4770-95b6-665508134d86 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:49.700400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 19:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:49.706758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e615e4a-3df4-4057-a358-6e9e5d04d363 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:49.707225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e615e4a-3df4-4057-a358-6e9e5d04d363 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:49.707367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e615e4a-3df4-4057-a358-6e9e5d04d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:49.727154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4e615e4a-3df4-4057-a358-6e9e5d04d363 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:49.728381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4e615e4a-3df4-4057-a358-6e9e5d04d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:49.736346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:24:49.887681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6de96-e63b-4770-95b6-665508134d86 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:49.902939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6de96-e63b-4770-95b6-665508134d86 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:49.908467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:49.909069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:49.909354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:49.909634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:49.918451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:49.920079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3632f42-0287-4c06-9b40-f37ac44be673 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:49.929644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 767/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:53.961814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:53.962369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:53.962369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:53.963745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:53.969174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8d6a-998f-48fa-b12a-1dcb27628e9d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:53.969588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8d6a-998f-48fa-b12a-1dcb27628e9d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:53.969725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8d6a-998f-48fa-b12a-1dcb27628e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:53.988790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7aca8d6a-998f-48fa-b12a-1dcb27628e9d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:53.989969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7aca8d6a-998f-48fa-b12a-1dcb27628e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:53.998246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:54.290388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:54.292131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:54.372737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b167b9-b12b-428c-9366-630b15eb0061 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:54.376318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:54.376924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:54.377211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:54.377491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:54.386993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:54.388443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aba9c-afea-4f43-91ca-06861f61e1bc service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:54.398326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 762/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:24:56.544958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610820-6c61-4e74-8297-23defab866f4 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:56.545518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610820-6c61-4e74-8297-23defab866f4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:56.545518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610820-6c61-4e74-8297-23defab866f4 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:56.546867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:24:56.552045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8611-8eaf-433e-ad2e-574217b4b140 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:56.552437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8611-8eaf-433e-ad2e-574217b4b140 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:56.552609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8611-8eaf-433e-ad2e-574217b4b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:56.573159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ea4d8611-8eaf-433e-ad2e-574217b4b140 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:24:56.574360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ea4d8611-8eaf-433e-ad2e-574217b4b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:24:56.582144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:24:56.867769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610820-6c61-4e74-8297-23defab866f4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:24:56.881415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610820-6c61-4e74-8297-23defab866f4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:24:56.885780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:24:56.889318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:24:56.889318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:24:56.889318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:24:56.899334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:24:56.900854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e06381-260e-4f13-a1f0-c327b6380e0b service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:24:56.910923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 770/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:25:00.726948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:00.727481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:00.727481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:00.728883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:25:00.734304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04ad5-d769-4bd7-9fca-69e002840224 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:00.734718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04ad5-d769-4bd7-9fca-69e002840224 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:00.734864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04ad5-d769-4bd7-9fca-69e002840224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:00.754405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dad04ad5-d769-4bd7-9fca-69e002840224 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:25:00.755553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dad04ad5-d769-4bd7-9fca-69e002840224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:25:00.763990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:25:01.066746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:25:01.068456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:25:01.133906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:25:01.146781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901932e-16aa-4a1c-800d-dd5297513229 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:25:01.151012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:25:01.151619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:01.151942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:01.152220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:01.161267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:25:01.162695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebbb90-dd7f-477e-acab-926b8223d453 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:25:01.172881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 765/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:25:03.287760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:03.288392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:03.288392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:03.289692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:25:03.294915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ecb5-2bf3-4b83-98b5-9b39721848f9 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:03.295327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ecb5-2bf3-4b83-98b5-9b39721848f9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:03.295469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890ecb5-2bf3-4b83-98b5-9b39721848f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:03.315374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d890ecb5-2bf3-4b83-98b5-9b39721848f9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:25:03.316594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d890ecb5-2bf3-4b83-98b5-9b39721848f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:25:03.324999 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:25:03.624763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:25:03.626450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:25:03.639817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eef1a4-87e6-4b8b-bc78-531205f01457 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:25:03.643285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:25:03.643909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:25:03.644193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:25:03.644471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:25:03.654545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:25:03.656007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45380aa-06e9-4ff8-8ce9-761630f23550 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:25:03.665296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 773/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:25:13.310649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=11 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:25:13.661166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:27.283471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.284163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.284163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.313012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.314361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.324439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 767/1540] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:26:27.333504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.334090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.334090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.359331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.363779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.375207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 774/1541] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:27.380579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:27.393252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae90ea0-7cf8-48fd-ab8f-02eb25150f73 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:27.404158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:27.404852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.405127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.405406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.406067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:27.406372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:27.408381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2197-7e13-47ca-98d7-53d97c4467ec admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:27.430784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 768/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:27.439358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:27.453902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc9455-3996-4ec5-a26f-1e7693f5a85e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:27.463611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:27.467226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.467682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.467975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.468642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:27.470582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:27.536552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742055b-1382-4bc2-ae7f-b1f9327cbfbe None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.536762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742055b-1382-4bc2-ae7f-b1f9327cbfbe None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.537024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742055b-1382-4bc2-ae7f-b1f9327cbfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.573732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1742055b-1382-4bc2-ae7f-b1f9327cbfbe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.577927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1742055b-1382-4bc2-ae7f-b1f9327cbfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.588765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 769/1543] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:27.597461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.597784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.598032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.629397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.630817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.643303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 770/1544] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:26:27.648745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:27.658854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2f816-3734-4205-b80a-25ddd13f710f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:27.673112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 775/1545] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:27.676771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:27.785540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08599f0-2189-414a-aa13-fbf1259bb3d1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:27.788672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:27.789253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.789639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.789926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.790699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:27.791017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:27.792765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efc0b-6eca-4855-8f19-c75b9ec1fdde admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:27.795296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:27.795971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.796260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.796597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.797253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:27.798271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:27.798271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:26:27.798271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:26:27.799490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be388e-5198-4ebf-a765-3154d9a3a5e2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:27.817479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 771/1546] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:26:27.818801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470388-2a34-472b-af96-3530dbc8d03f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.819095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470388-2a34-472b-af96-3530dbc8d03f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.819377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470388-2a34-472b-af96-3530dbc8d03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.861888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3f470388-2a34-472b-af96-3530dbc8d03f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.863130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3f470388-2a34-472b-af96-3530dbc8d03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.876271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 772/1547] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:27.878930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.878930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.878930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.913440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.914742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.926916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 773/1548] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:27.928140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:27.941311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949bf8-4c29-45f2-8165-3b520f084f2b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:27.944864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 776/1549] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:27.946370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52088e7f-90ea-4b55-adc6-4cd338a94d60 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.946882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52088e7f-90ea-4b55-adc6-4cd338a94d60 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.947371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52088e7f-90ea-4b55-adc6-4cd338a94d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.947492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:27.948223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.948518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:26:27.952697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cbaf2-77c5-4b08-a666-9a6be1b2422c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:27.975618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-52088e7f-90ea-4b55-adc6-4cd338a94d60 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:27.977143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-52088e7f-90ea-4b55-adc6-4cd338a94d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:27.985749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 777/1550] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:27.986987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8151a4-e765-4327-a6d8-3aa6a7590385 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:27.987268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8151a4-e765-4327-a6d8-3aa6a7590385 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:27.987539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8151a4-e765-4327-a6d8-3aa6a7590385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.008229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3f8151a4-e765-4327-a6d8-3aa6a7590385 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.011821 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f8151a4-e765-4327-a6d8-3aa6a7590385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.017301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 778/1551] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.018535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.018820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.019094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.037917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.040609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.047542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 779/1552] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.048368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.058950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170113-f7b9-434c-8b7d-323f4666e324 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.063904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.070612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.070612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.070612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.070612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.070940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 774/1553] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:27 2022] POST /identity/v3/projects => generated 359 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:28.071033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc00234-786b-4e98-aa63-60c7287eebb0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.071123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51c43-57bf-47fc-93eb-9074fab82d9a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.071123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51c43-57bf-47fc-93eb-9074fab82d9a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.071123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51c43-57bf-47fc-93eb-9074fab82d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.078976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 780/1554] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:28.080109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d7cd-b537-454c-82f9-81b4ef9ffd5b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.080403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d7cd-b537-454c-82f9-81b4ef9ffd5b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.081244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d7cd-b537-454c-82f9-81b4ef9ffd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.092728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9ba51c43-57bf-47fc-93eb-9074fab82d9a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.093951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ba51c43-57bf-47fc-93eb-9074fab82d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.100780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2005d7cd-b537-454c-82f9-81b4ef9ffd5b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.101813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2005d7cd-b537-454c-82f9-81b4ef9ffd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.156198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 775/1555] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.161151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.161349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.161662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.163112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 781/1556] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.164460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09216-51d9-4a93-9b85-1d567e22d09f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.164676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09216-51d9-4a93-9b85-1d567e22d09f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.164955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09216-51d9-4a93-9b85-1d567e22d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.183956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.185226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.185373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-abe09216-51d9-4a93-9b85-1d567e22d09f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.186530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-abe09216-51d9-4a93-9b85-1d567e22d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.193656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 776/1557] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.194883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.197332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 782/1558] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.198852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.199294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.199826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.207157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6492b3-e379-41db-aa58-d7d704cadcc6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.213022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.213634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.213919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.214195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.214843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.215152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.216853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.220959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.222262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.228815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 777/1559] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.229592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.230769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 783/1560] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.231390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.241552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9aa62-daf6-488b-8761-5eb1bd5115b8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.244846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76de0-5e90-43f2-905c-f9f76306fe93 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.246244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.246836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.247121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.247399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.248087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.249795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.249931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.250391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.250674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.250955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.251585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.252098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:28.253522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.261022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 784/1561] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:28.261853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.262389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:28.271565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022df1-b180-4324-9b3f-b2358e652736 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.276422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.277006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.277298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.277578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.278251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.278560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.279745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:28.279745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:26:28.279745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:26:28.280625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf1657-c046-422c-9128-08211e35f1d9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.293334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 785/1562] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.294434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.294722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.294999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.314889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.316113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.324781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 786/1563] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.325631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.334960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99acfcc-8b8d-43ef-bb85-dd2711491014 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.339730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.340347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.340633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.340910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.341539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.341845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.343594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ecb-729c-43f8-833d-b8e04bbd0031 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.355727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 787/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.356782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.357156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 778/1565] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/users => generated 280 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:28.357809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.360129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.360427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.368901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964d36f-a291-445c-b39f-237c1617ca70 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.373943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.374558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.374844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.375150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.375828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.377803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.381170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:28.382426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:28.390379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 788/1566] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:28.391185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.400512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9899-d7c1-4092-b39b-260fa6c4517e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.405296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.405900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.406183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.406459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.407098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.407402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.409039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.420972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 789/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.421750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.433010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a8b3-9b14-4664-af42-7e58e4a8a1b5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.437870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.438616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.438904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.439185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] PATH_INFO: `/v3/projects/4a866fb1264140c1a1056696bf6fabc7/users/1ab4dbb6c98f4c99bbb06f586aaddeba/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.454504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a866fb1264140c1a1056696bf6fabc7, user_id=1ab4dbb6c98f4c99bbb06f586aaddeba, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.457354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.513351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 779/1568] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:28.514507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.549957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 790/1569] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:28 2022] PUT /identity/v3/projects/4a866fb1264140c1a1056696bf6fabc7/users/1ab4dbb6c98f4c99bbb06f586aaddeba/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:28.549957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.594170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd9ce-4fab-46cf-9432-13fdeb0efe39 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.624418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424350c-fc78-4d8c-b143-ef6cbfbc1ccc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.630080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.630267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.630412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.630685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.631851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.632158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.633784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.648572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 791/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.649718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.658046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.658819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.659104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.659382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.660118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.661238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5104-db01-4f74-b6cb-844578123b43 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.662406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.665929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.666533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.666823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.667100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.667773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.669383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.681006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 792/1571] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:28.683370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.692881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4d025-620e-47cb-93bd-66ff415f5cda admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.700613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.701290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.701570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.701844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.702482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.704551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.759483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 780/1572] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/projects => generated 365 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:28.760451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.825702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 793/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:28.825702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.830411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03b246-15f2-48c1-badf-861b0d994c74 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.834971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:28.870674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:28.888279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb600a-33c2-4409-bc30-ac5cc773a38d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.889192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.889811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.890159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.890443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.891096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:28.891406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.893043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.893314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.893921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.894202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.894537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.895167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.897301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c004d1-3669-4d9c-a85b-21bb3a1ec8fa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:28.906015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 781/1574] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:28.906843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:28.916932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb8cc0-0ce9-47d6-8845-865e1398550c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:28.921719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:28.922333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:28.922623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:28.922914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:28.923539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:28.925548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.009148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 794/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.010662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.010933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.011211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.030145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:29.031359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:29.036621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 782/1576] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:28 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.037965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.048713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5b4f-e52e-456f-beb1-7d3e7e05ed73 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.104488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 795/1577] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:29.105381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.110228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.110978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.111405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.111807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.112542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.115126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.115290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339469a-6396-4343-9f7c-1c6afd0301e9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.121164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.121486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.121766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.122043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.122713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.124362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.134647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 796/1578] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.135632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.145563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4411f42-3956-487a-8559-ed4eb1c68294 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.150327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.150907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.151192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.151467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.152136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.154113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.197232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 783/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.198066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.261018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f7a95-a117-495f-9818-3a18eccfe5df admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.265836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.266532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.266933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.268516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.268516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.271243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.273474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 797/1580] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.274288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.285202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71da0-4154-4945-a723-9ddc65bf8231 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.291887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.292509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.292795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.293074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.293715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.295614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.369121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:29.375083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 784/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/users => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.375944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.385966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f0f1-f5bd-4853-99e5-80ea10edfe7d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.387473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 798/1582] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.388354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.450919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.451528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.451843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.452122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.452756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.454655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.457031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c16e2f-4027-48ad-b24b-f811fad5c2af admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.462471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.463087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.463365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.463640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.464317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.465969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.477081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 799/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.477943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.491565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:29.533770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9e498-a337-48b6-a4f6-9a8163406912 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.537418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.538905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.538905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.539204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.548163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.549582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.559201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 800/1584] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:29.561902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.569042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94a26-0920-44f7-904d-14c56ce97417 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.574205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.574947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.575293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.575593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] PATH_INFO: `/v3/projects/f1404906e54a4b85b96f8270984d55bf/users/0931d884ab9d497bab35a2725fe1b4e9/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.589867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 785/1585] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.590883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.593695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1404906e54a4b85b96f8270984d55bf, user_id=0931d884ab9d497bab35a2725fe1b4e9, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.596506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.601397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23f9c-929b-48d5-9be6-0ef1be2a31ca admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.605939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.606540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.606817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.607092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.607757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.609638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.713394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 786/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.714487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.726616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 801/1587] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:29 2022] PUT /identity/v3/projects/f1404906e54a4b85b96f8270984d55bf/users/0931d884ab9d497bab35a2725fe1b4e9/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:29.726616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.742440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573f89c-b491-4494-86a4-d9a7f77cce4e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.790105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9b300-c533-47ee-8997-d164e42377ea admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.794981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.799729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.799729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.799729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.799729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.799729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.808635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.809137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 802/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.809466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.809860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.809982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.810340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.811143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.813185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.822204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c44e1-7278-4ded-9468-557c888c56ee admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.824150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 787/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.824952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.830239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.830859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.831145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.831693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.832093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.834030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.838009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963618-36f4-4368-8484-71469a865ca1 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.843257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.845504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.845504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.845504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.845504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.846871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.857818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 788/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.858631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.868261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d9c5a-ac50-4e13-957f-5787acbd974c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.872874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.873495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.873772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.874052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.874684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.880109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.890650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 789/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.890650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.899503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7291ab-5ea4-4f76-a8d2-55f3e67d567d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.905005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.909364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.909364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.909364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.909364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.909364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372450c-a98c-4635-9921-66a840a6b49a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.925905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 790/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.927431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.927833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.928220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.945900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 803/1593] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:29.945900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:29.968098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] MFA Rules not processed for user `0931d884ab9d497bab35a2725fe1b4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:29.977984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384878e-47ad-4312-8b5f-a511456b8adf admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:29.978168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:29.978269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:29.978269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:29.978269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:29.978269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:29.978634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:29.979335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:29.993941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:29.997455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 804/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:29.998476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.010038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb73c3-b5d7-44eb-bf7e-0a0f6b7b5a01 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.014790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.017982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.017982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.017982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.017982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.018435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.029863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 805/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:30.030990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.042998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a63f7-2ad7-4846-acb0-e3d493115889 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.047930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.048640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.048929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.049208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] PATH_INFO: `/v3/projects/016f968319fd49d98506030c0bee6021/users/1d7c40b5232e46119434aeeb8a6ad4d0/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.052173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 791/1596] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:30.052894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.064263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db29cd6-9cd5-4464-bed5-daed718e5f49 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.069061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] RBAC: Authorizing `identity:create_grant(project_id=016f968319fd49d98506030c0bee6021, user_id=1d7c40b5232e46119434aeeb8a6ad4d0, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.069830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.070515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.071183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.071524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.072247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.072538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.075136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.085784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 792/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:30.086620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.096243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a89b-90ee-4579-8bdc-f9849f3112fa admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.101463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.102070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.102355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.102626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.103256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.104858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.114835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 793/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:30.115722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.130466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fb4ed-c189-460a-9f3d-1b6d1377b891 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.135782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.136519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.136847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.137321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.138270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.140048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.154245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 794/1599] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:30.154245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.164530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 806/1600] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/016f968319fd49d98506030c0bee6021/users/1d7c40b5232e46119434aeeb8a6ad4d0/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:30.164817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c5fb-7ef9-42f2-8733-028f1196c287 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.165852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.201803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:30.230296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.231060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.231500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.231859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] PATH_INFO: `/v3/projects/0eddbf81997c4b95893ee57a21200213/users/2da94a3595d548e79c31eba282d03455/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.244090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b46d-af94-4d0d-88b9-4a937a59617f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.249185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.249870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.251004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.251169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eddbf81997c4b95893ee57a21200213, user_id=2da94a3595d548e79c31eba282d03455, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.251297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] PATH_INFO: `/v3/projects/73be7c05690c456cbee1481590453dd5/users/07689aa819d8466394c59d8044543c30/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.251506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:30.251506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:26:30.251506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:26:30.253475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.266908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73be7c05690c456cbee1481590453dd5, user_id=07689aa819d8466394c59d8044543c30, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.270025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.406804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 795/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/0eddbf81997c4b95893ee57a21200213/users/2da94a3595d548e79c31eba282d03455/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:30.406804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.407592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 807/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/73be7c05690c456cbee1481590453dd5/users/07689aa819d8466394c59d8044543c30/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:30.407592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.486706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c12d7-2eb6-4b99-9ca9-aa695d817043 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.489691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c65cf-906b-401f-ab8d-eacdcc791e21 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.491772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.492511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.492795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.493073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] PATH_INFO: `/v3/projects/ecc8dce537dd4fb2901ca3b2b62a5dc2/users/121295939fc44468bd2b04ed47f45886/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.495770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.496493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.496856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.497207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.497951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.499946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.509429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc8dce537dd4fb2901ca3b2b62a5dc2, user_id=121295939fc44468bd2b04ed47f45886, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.512541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.607157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 808/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/ecc8dce537dd4fb2901ca3b2b62a5dc2/users/121295939fc44468bd2b04ed47f45886/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:30.608005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.630198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:30.638164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 796/1604] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/projects => generated 365 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:30.639071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.681772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1211fee-8ce3-4886-8e4e-b9dae2d4d6ef admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.712795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664994f-39e9-48b3-9c1d-d502459455a6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:30.719829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:30.721148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.721693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.721985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.722264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.723104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.725673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1d6bc-4123-4c88-892d-270f4d2cd5a1 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.741788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 797/1605] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:30.743307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16036162-131c-4723-b1e8-04d0a1947b80 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.744583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16036162-131c-4723-b1e8-04d0a1947b80 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.744896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16036162-131c-4723-b1e8-04d0a1947b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.754536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:30.754536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.754536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.754536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] PATH_INFO: `/v3/projects/064ccd91385d4f7db985421d083e4b95/users/9823a946acd148f0baf138b53bc1edf7/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.764790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064ccd91385d4f7db985421d083e4b95, user_id=9823a946acd148f0baf138b53bc1edf7, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:30.766563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-16036162-131c-4723-b1e8-04d0a1947b80 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:30.767837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-16036162-131c-4723-b1e8-04d0a1947b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:30.768075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7aba-f71d-45e1-9b30-f22990d975e0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:30.775505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 798/1606] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:30.776764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973104c5-12e8-4f38-89a2-9b72cb4fceae None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.784068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973104c5-12e8-4f38-89a2-9b72cb4fceae None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.784068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973104c5-12e8-4f38-89a2-9b72cb4fceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.807292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-973104c5-12e8-4f38-89a2-9b72cb4fceae None None] MFA Rules not processed for user `2da94a3595d548e79c31eba282d03455`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:30.808574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-973104c5-12e8-4f38-89a2-9b72cb4fceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:30.843996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 809/1607] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/064ccd91385d4f7db985421d083e4b95/users/9823a946acd148f0baf138b53bc1edf7/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:30.845469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.845857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.846232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.879461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] MFA Rules not processed for user `07689aa819d8466394c59d8044543c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:30.880015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 799/1608] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:30.880981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:30.881303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:30.881592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:30.881892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:30.911896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] MFA Rules not processed for user `121295939fc44468bd2b04ed47f45886`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:30.913588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:30.962691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 810/1609] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:30.963558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:30.992883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 800/1610] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:30.993933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.004369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:31.060115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ae1d-2113-4b8a-901b-f13641d811db None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.067021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.068213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.068507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.068813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.069464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.071513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.086448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5d44-c5ac-4208-8053-d26d9de67dff None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.091481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.092380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.092761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.093046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] PATH_INFO: `/v3/projects/53ec78c2e65c4f0c86d505c409031ab6/users/0e37c8ca70ad4b6e8430a71d0f5e564f/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.110205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ec78c2e65c4f0c86d505c409031ab6, user_id=0e37c8ca70ad4b6e8430a71d0f5e564f, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.113405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa7c3b-459b-487b-be51-3ae21f71bdc2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.158514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 811/1611] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:30 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:31.159887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.163958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:31.189324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 801/1612] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:30 2022] PUT /identity/v3/projects/53ec78c2e65c4f0c86d505c409031ab6/users/0e37c8ca70ad4b6e8430a71d0f5e564f/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:31.192088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.192088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.192088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.227387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:31.234351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] MFA Rules not processed for user `9823a946acd148f0baf138b53bc1edf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:31.235589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:31.245924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21dc-ad8a-4647-97ff-bd0a0db1cda6 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.304979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.305662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.305963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.306368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.324643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 802/1613] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:31.325591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.402132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad6d1b-5cd8-474c-808f-5c053ff5e76c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.406039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.408075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.408688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.409187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.409730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.411457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.411457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.413863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6556af-0ab0-48e5-946e-7f315fb62242 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.419580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 812/1614] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:31.420737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.432159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63303301-ba5d-4eaf-ad98-312ea8de38cd service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.438541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.439169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.439453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.439767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.440434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.442479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7431-4fa5-46d4-b7c8-f13018c6027c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.527901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 803/1615] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/projects => generated 403 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:31.527901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0976d-02fc-4f48-8c59-736098f3af53 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.527901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0976d-02fc-4f48-8c59-736098f3af53 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.527901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0976d-02fc-4f48-8c59-736098f3af53 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.528747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 804/1616] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:31.529384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0976d-02fc-4f48-8c59-736098f3af53 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.535627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:31.563475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 813/1617] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:31.564962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d5d0-c32c-41b6-9ef1-46b9638512b0 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.565255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d5d0-c32c-41b6-9ef1-46b9638512b0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.565534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d5d0-c32c-41b6-9ef1-46b9638512b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.587867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7975d5d0-c32c-41b6-9ef1-46b9638512b0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:31.589566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7975d5d0-c32c-41b6-9ef1-46b9638512b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:31.604517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0976d-02fc-4f48-8c59-736098f3af53 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.672370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 814/1618] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:31.673564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d5d8-f6d9-4520-a984-e6bb6e272125 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.674030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d5d8-f6d9-4520-a984-e6bb6e272125 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.674706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d5d8-f6d9-4520-a984-e6bb6e272125 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.676578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 815/1619] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 15 19:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:31.677386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7d5d8-f6d9-4520-a984-e6bb6e272125 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.694665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7d5d8-f6d9-4520-a984-e6bb6e272125 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.700657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.701267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.701569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.701848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.702532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.704590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.722009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.722665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.723251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.723251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.723896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.726093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c16df-b78a-4717-8951-8aadd6bc79aa admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.740736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 805/1620] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:31.742954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.743376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.743798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.788431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:31.789670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:31.835534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 816/1621] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/users => generated 308 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:31.837407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.852674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f974e3c-3981-420b-90fe-76bb5134d439 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.858894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.858894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.859377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.859763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.860486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.862657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563cf07-8add-42d4-a4ae-0038b1d46154 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.867325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 806/1622] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:31.867579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:31.935205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103eae-76ec-4341-b822-6f7bca0b9b29 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:31.939420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:31.941798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.941798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.941798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.951805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:31.953558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:31.970371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 817/1623] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:31.972471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:31.972471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:31.972471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:31.981738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 807/1624] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:31.989229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.004755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:32.004967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c1-5881-421f-8ba4-ea3c2df0e51f service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.005096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:32.011142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.012307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.012511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.012843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] PATH_INFO: `/v3/projects/dd39254ccc1b454e82972a58b9c471bf/users/1ff7a96ff0cc4ead9fe25835d0546744/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.018080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 818/1625] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 15 19:26:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:32.019566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.034423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9771202-6a41-4c85-b559-fca866e31cf3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.040577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.041430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.041723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.042003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.042817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.044673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.045859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd39254ccc1b454e82972a58b9c471bf, user_id=1ff7a96ff0cc4ead9fe25835d0546744, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.048917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.068321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 819/1626] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:32.069606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.090616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c4d6-e3bd-406e-8146-53e8158497cd admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.098370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.098997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.099283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.099591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.170849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:32.179183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 808/1627] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:31 2022] PUT /identity/v3/projects/dd39254ccc1b454e82972a58b9c471bf/users/1ff7a96ff0cc4ead9fe25835d0546744/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:32.179774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.298709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.298709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.298709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 820/1628] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:32.298709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.298709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2698629-58e3-4ec2-9021-20213d043179 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.361295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7984a5-f41c-40d8-a4a9-2ba9e499f539 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.368305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.369219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.369668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.370126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.371034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.372831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.389718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 809/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:32.394196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.406007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f33dc-68a8-4f96-8fc9-02a8d37f40d6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.411831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.413600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.414053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.414517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.415286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.416158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.416602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.416945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] PATH_INFO: `/v3/projects/f2d7c923153f4e01aceb606eca92ec8f/users/2d43a65829e04e80a8696482a8f677a9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.443152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d7c923153f4e01aceb606eca92ec8f, user_id=2d43a65829e04e80a8696482a8f677a9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.456070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.509371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.513114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.525466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 821/1630] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:32.526869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.541597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453e7f6-785c-4242-a2e7-bdc9724d55f1 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.547758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.548743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.549120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.549533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.550402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.552944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.573775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 810/1631] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:32 2022] PUT /identity/v3/projects/f2d7c923153f4e01aceb606eca92ec8f/users/2d43a65829e04e80a8696482a8f677a9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:32.574423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.601299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 822/1632] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:32.605711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.632302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:32.637587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b218fbe-4eac-4a5f-9117-c607487e32f5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.741849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7251-60fb-4832-aaa0-599f18c79d9e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.755916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.757049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.757613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.758151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] PATH_INFO: `/v3/projects/6d4cda75c68d400f817239f82595f869/users/b8cc749441a14eafbe31aa106a326b60/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.760251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.760905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.761181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.761519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.762420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.764444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.786856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 823/1633] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:32.786856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.801652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc231e3-07f7-472a-bf45-bb8579cb41d0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:32.808364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4cda75c68d400f817239f82595f869, user_id=b8cc749441a14eafbe31aa106a326b60, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.810050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:32.810963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.811308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.811734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] PATH_INFO: `/v3/projects/dd39254ccc1b454e82972a58b9c471bf/users/1ff7a96ff0cc4ead9fe25835d0546744/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.814327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd39254ccc1b454e82972a58b9c471bf, user_id=1ff7a96ff0cc4ead9fe25835d0546744, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:32.817891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26faa73c-e7e8-4c6d-9162-f05b9aa64e64 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.819185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3118c-7c4e-4a61-b618-50911a9dd21c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:32.869487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 824/1634] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:32 2022] PUT /identity/v3/projects/dd39254ccc1b454e82972a58b9c471bf/users/1ff7a96ff0cc4ead9fe25835d0546744/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:32.873577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc865-119e-4d71-bdcd-a84ceb0ed102 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.873881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc865-119e-4d71-bdcd-a84ceb0ed102 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.875420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc865-119e-4d71-bdcd-a84ceb0ed102 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:32.878846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 825/1635] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 15 19:26:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:32.880975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadc865-119e-4d71-bdcd-a84ceb0ed102 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:32.958067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 811/1636] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:32 2022] PUT /identity/v3/projects/6d4cda75c68d400f817239f82595f869/users/b8cc749441a14eafbe31aa106a326b60/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:32.962487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67e6f2-2825-4305-bdf1-5213ddce4ff8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:32.962487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67e6f2-2825-4305-bdf1-5213ddce4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:32.962487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67e6f2-2825-4305-bdf1-5213ddce4ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.052039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadc865-119e-4d71-bdcd-a84ceb0ed102 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.052860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7c67e6f2-2825-4305-bdf1-5213ddce4ff8 None None] MFA Rules not processed for user `1ff7a96ff0cc4ead9fe25835d0546744`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:33.055144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7c67e6f2-2825-4305-bdf1-5213ddce4ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:33.177192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 812/1637] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:33.180868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f13250-bc1e-4d3b-9aeb-0fcacd255680 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.191011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f13250-bc1e-4d3b-9aeb-0fcacd255680 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.191419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f13250-bc1e-4d3b-9aeb-0fcacd255680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.194340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.195102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.203005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.203005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] PATH_INFO: `/v3/projects/f2d7c923153f4e01aceb606eca92ec8f/users/2d43a65829e04e80a8696482a8f677a9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.515736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d7c923153f4e01aceb606eca92ec8f, user_id=2d43a65829e04e80a8696482a8f677a9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:33.515736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:33.525787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-41f13250-bc1e-4d3b-9aeb-0fcacd255680 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:33.525787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-41f13250-bc1e-4d3b-9aeb-0fcacd255680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:33.528954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 826/1638] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:32 2022] PUT /identity/v3/projects/f2d7c923153f4e01aceb606eca92ec8f/users/2d43a65829e04e80a8696482a8f677a9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:33.528954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:33.528954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd811-6e4b-4a22-a5ed-a2d477e3da74 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.528954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.530629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.530629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.530629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.530629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:33.531190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:33.531469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:33.531698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 813/1639] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:33.531698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596cb8-f3f5-4c14-b607-522936a44bda None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.532203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 827/1640] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:33.532203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596cb8-f3f5-4c14-b607-522936a44bda None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596cb8-f3f5-4c14-b607-522936a44bda None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 814/1641] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 15 19:26:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.532605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.533434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] MFA Rules not processed for user `2d43a65829e04e80a8696482a8f677a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:33.533434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:33.552121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c3d88-7127-4e71-9042-b5ce58763fe8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.568627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.570482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.570945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.571391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.623379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 815/1642] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:33.624666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:33.637758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4382be8-140c-4240-8c95-6bd8bfbf630a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.644139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.645041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.645457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.645740 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] PATH_INFO: `/v3/projects/6d4cda75c68d400f817239f82595f869/users/b8cc749441a14eafbe31aa106a326b60/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.648937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4cda75c68d400f817239f82595f869, user_id=b8cc749441a14eafbe31aa106a326b60, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:33.651161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0a2f8-980d-4d4b-9485-e2049e128795 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:33.658165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:33.659841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:33.663542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 816/1643] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:33 2022] PUT /identity/v3/projects/6d4cda75c68d400f817239f82595f869/users/b8cc749441a14eafbe31aa106a326b60/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:33.664783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abad27-9bcc-4bce-8f64-82430f94c938 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.665246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abad27-9bcc-4bce-8f64-82430f94c938 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.665627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abad27-9bcc-4bce-8f64-82430f94c938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.670305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 828/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:33.671415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:33.682730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3f9c-5140-48aa-88ff-59aa87945cd5 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.690369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-08abad27-9bcc-4bce-8f64-82430f94c938 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:33.691849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-08abad27-9bcc-4bce-8f64-82430f94c938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:33.743365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:33.770409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.773754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.774102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.774393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.788367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 817/1645] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:33.791638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.792352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.793085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:33.840104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] MFA Rules not processed for user `b8cc749441a14eafbe31aa106a326b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:33.841349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:33.907080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:33.909515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:33.947856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 829/1646] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:33.949039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:33.971592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fb718-8991-48e4-8899-4fd789f39ffe service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:33.980065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:33.983178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:33.983686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:33.984406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:34.017521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 818/1647] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:34.092403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:34.094329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:34.111994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 830/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:34.204141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:34.222092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086aa69-f074-417c-bf77-056bd914ef44 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:34.230127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:34.232231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:34.232726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:34.233219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:34.253608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:34.258320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:34.296708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 819/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:34.430897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:34.449237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb25c2-2ab7-46f2-af36-7d4e20215e5e service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:34.455426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:34.456031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:34.456382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:34.456821 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:34.482708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:34.484159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:34.494644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 831/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:34.967015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:35.126367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:35.128595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:35.152868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14527017-1970-461b-b9f6-0caf759a49e1 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:35.165913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:35.166546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:35.166950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:35.167121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:35.212112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:35.340256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:35.347873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:35.394578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 832/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:35.395944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:35.444809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58123d-1039-4e76-99c8-0a769abaee34 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:35.472659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:35.473376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:35.473795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:35.473990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] PATH_INFO: `/v3/users/9823a946acd148f0baf138b53bc1edf7` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:35.488255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9823a946acd148f0baf138b53bc1edf7)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:35.489400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: WARNING py.warnings [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:35.489400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: warnings.warn(msg) Sep 15 19:26:35.489400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: Sep 15 19:26:35.503451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:35.590887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71905-92c1-4fcf-9f39-0e0000ea98e9 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:35.616145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:35.617190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:35.617516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:35.617861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] PATH_INFO: `/v3/users/2da94a3595d548e79c31eba282d03455` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:35.621999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da94a3595d548e79c31eba282d03455)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:35.629416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:35.660653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:35.948980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da94a3595d548e79c31eba282d03455'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:35.988148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9823a946acd148f0baf138b53bc1edf7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.049581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da94a3595d548e79c31eba282d03455'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.050336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da94a3595d548e79c31eba282d03455'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.051043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da94a3595d548e79c31eba282d03455'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.064053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a16c1a4-854a-4368-b6bf-9200c0376aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da94a3595d548e79c31eba282d03455'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.112569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 833/1652] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:35 2022] DELETE /identity/v3/users/2da94a3595d548e79c31eba282d03455 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:36.114063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:36.114382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:36.114701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:36.140304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:36.141333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:36.169205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9823a946acd148f0baf138b53bc1edf7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.170451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9823a946acd148f0baf138b53bc1edf7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.172202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9823a946acd148f0baf138b53bc1edf7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.216598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-14fca21f-f2ee-40c2-a1a0-36a7bb0eac69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9823a946acd148f0baf138b53bc1edf7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:36.230304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 834/1653] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:36.237988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:36.241093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:36.242375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 820/1654] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:35 2022] DELETE /identity/v3/users/9823a946acd148f0baf138b53bc1edf7 => generated 0 bytes in 1117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:36.247596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:36.248032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:36.248371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:36.298995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:36.304993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:36.414827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ccf2-dbb9-476b-ba4b-58421072d1b4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:36.505252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 821/1655] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:36.524354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:36.527148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:36.539865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:36.593161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:36.593816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:36.594083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:36.594359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:36.650866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:36.655104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:36.677357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 835/1656] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:36.757905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ca4ff-38ed-4c1a-b740-273e17a2292e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:36.764447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:36.765673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:36.766285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:36.766926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:36.785121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:36.787813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81127a-38c2-41f2-9e2f-4f2ca3148c30 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:36.809367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 822/1657] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:26:37.009275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:37.016895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:37.047375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553225-a109-44ea-bfc9-eceae18776d3 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:37.082874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:37.083556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:37.083891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:37.087113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] PATH_INFO: `/v3/users/0931d884ab9d497bab35a2725fe1b4e9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:37.094742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0931d884ab9d497bab35a2725fe1b4e9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:37.114211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:37.465424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:37.517790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0931d884ab9d497bab35a2725fe1b4e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:37.637813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0931d884ab9d497bab35a2725fe1b4e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:37.638509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0931d884ab9d497bab35a2725fe1b4e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:37.639346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0931d884ab9d497bab35a2725fe1b4e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:37.651034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0931d884ab9d497bab35a2725fe1b4e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:37.714097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 836/1658] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:37 2022] DELETE /identity/v3/users/0931d884ab9d497bab35a2725fe1b4e9 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:37.717862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:37.718400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:37.718469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:37.764867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:37.778269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:37.791350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:37.920834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 823/1659] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:38.073084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:38.075392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:38.223604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:38.236627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84e8b7-51ed-4167-bd61-e4425bf471e7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:38.246305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:38.247471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:38.247892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:38.248243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:38.276588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:38.285037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:38.302412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 837/1660] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:38.768640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:38.770938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:38.811617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a0ab7-a964-4889-a257-21b350273105 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:38.821632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:38.827012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:38.827319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:38.828688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] PATH_INFO: `/v3/projects/064ccd91385d4f7db985421d083e4b95` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:38.830534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=064ccd91385d4f7db985421d083e4b95)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:38.834700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:38.855245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:38.857568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:38.895147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4662ee0-f9e5-4ad2-a322-c921e85973a3 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:38.935151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:38.936031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:38.936313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:38.936637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] PATH_INFO: `/v3/projects/0eddbf81997c4b95893ee57a21200213` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:38.941863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eddbf81997c4b95893ee57a21200213)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:38.942931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: WARNING py.warnings [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 19:26:38.942931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: warnings.warn(msg) Sep 15 19:26:38.942931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: Sep 15 19:26:38.944623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:39.110261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064ccd91385d4f7db985421d083e4b95', 'request_id': 'req-e1daedaf-b35f-408d-9264-22fd32cec3e3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.137565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064ccd91385d4f7db985421d083e4b95', 'request_id': 'req-e1daedaf-b35f-408d-9264-22fd32cec3e3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.194186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eddbf81997c4b95893ee57a21200213', 'request_id': 'req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.215995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eddbf81997c4b95893ee57a21200213', 'request_id': 'req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.264790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064ccd91385d4f7db985421d083e4b95', 'request_id': 'req-e1daedaf-b35f-408d-9264-22fd32cec3e3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.285519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] The token cache is being invalidate because project 064ccd91385d4f7db985421d083e4b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:39.286521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.288426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.292714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 824/1661] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:38 2022] DELETE /identity/v3/projects/064ccd91385d4f7db985421d083e4b95 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:39.297069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:39.299063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:39.377988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eddbf81997c4b95893ee57a21200213', 'request_id': 'req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.408827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] The token cache is being invalidate because project 0eddbf81997c4b95893ee57a21200213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:39.410295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.413336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c758b3cb-a5e2-477f-99cd-ecd6d9c2a279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:39.422603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 838/1662] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:38 2022] DELETE /identity/v3/projects/0eddbf81997c4b95893ee57a21200213 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:26:39.422603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:39.424934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:39.425341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:39.448029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daedaf-b35f-408d-9264-22fd32cec3e3 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:39.482114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:39.487237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:39.534677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:39.535062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:39.535346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:39.535619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] PATH_INFO: `/v3/users/121295939fc44468bd2b04ed47f45886` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:39.542386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=121295939fc44468bd2b04ed47f45886)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:39.546762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:39.701491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 839/1663] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:39.702206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:39.704783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:39.731435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5d197-e3e7-4662-8d65-11dbe00715ed None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:39.740138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:39.741647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:39.741647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:39.741647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] PATH_INFO: `/v3/users/2d43a65829e04e80a8696482a8f677a9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:39.743450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d43a65829e04e80a8696482a8f677a9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:39.745826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:39.950053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '121295939fc44468bd2b04ed47f45886'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.046253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d43a65829e04e80a8696482a8f677a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.109880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '121295939fc44468bd2b04ed47f45886'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.110828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '121295939fc44468bd2b04ed47f45886'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.112339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '121295939fc44468bd2b04ed47f45886'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.147732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-cd088520-4346-4bfb-8199-856089cea6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '121295939fc44468bd2b04ed47f45886'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.200048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 825/1664] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:39 2022] DELETE /identity/v3/users/121295939fc44468bd2b04ed47f45886 => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:40.204001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7f51b-e603-4dd8-840a-870d79d84787 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:40.207181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7f51b-e603-4dd8-840a-870d79d84787 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:40.207715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7f51b-e603-4dd8-840a-870d79d84787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:40.276647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8fb7f51b-e603-4dd8-840a-870d79d84787 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:40.284329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8fb7f51b-e603-4dd8-840a-870d79d84787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:40.300368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d43a65829e04e80a8696482a8f677a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.301521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d43a65829e04e80a8696482a8f677a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.302293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d43a65829e04e80a8696482a8f677a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.322072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d43a65829e04e80a8696482a8f677a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.394137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 840/1665] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:39 2022] DELETE /identity/v3/users/2d43a65829e04e80a8696482a8f677a9 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:40.401742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:40.404576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:40.481119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 826/1666] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:40.485188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:40.485188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:40.485463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:40.518918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:40.521110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:40.571523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:40.611549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44523031-b8bf-4784-8e22-fe03ae5ef437 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:40.617851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:40.618514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:40.618806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:40.619146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] PATH_INFO: `/v3/projects/f1404906e54a4b85b96f8270984d55bf` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:40.619624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 827/1667] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:40.621767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:40.623676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1404906e54a4b85b96f8270984d55bf)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:40.632123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:40.647535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32952c1c-6489-4778-954a-c30ed895ec11 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:40.659637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:40.660491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:40.660807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:40.661150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:40.662034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:40.662835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:40.664806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:40.690275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 828/1668] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:40.694971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:40.707815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec31d-9737-4063-aae5-dfded290fa08 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:40.713807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:40.714602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:40.714913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:40.715568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:40.715977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:40.718079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:40.826063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1404906e54a4b85b96f8270984d55bf', 'request_id': 'req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.839973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1404906e54a4b85b96f8270984d55bf', 'request_id': 'req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:40.885945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 829/1669] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:40 2022] POST /identity/v3/projects => generated 403 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:40.886301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:40.893417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.003282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1404906e54a4b85b96f8270984d55bf', 'request_id': 'req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:41.024047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] The token cache is being invalidate because project f1404906e54a4b85b96f8270984d55bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:41.024731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:41.026228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:41.028462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 841/1670] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:40 2022] DELETE /identity/v3/projects/f1404906e54a4b85b96f8270984d55bf => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:41.029597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.033704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.052688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866550c-eba5-4cdc-b997-afb808ef9f92 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.067902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.069092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.069929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.070214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.076615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.210608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050db5-5265-4e36-822b-ee3b4ac0f3ab admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.241526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:41.278925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.280914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db584514-7d34-41ae-a9f3-fe8dd5c84ec0 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.298685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 830/1671] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:41.298685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.298685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.298685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.391205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] MFA Rules not processed for user `1ab4dbb6c98f4c99bbb06f586aaddeba`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:41.394711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.400572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85119663-a119-4e51-b091-79d591643aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.402295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119663-a119-4e51-b091-79d591643aad admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.402652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119663-a119-4e51-b091-79d591643aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.403005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119663-a119-4e51-b091-79d591643aad admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.403667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85119663-a119-4e51-b091-79d591643aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.409808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85119663-a119-4e51-b091-79d591643aad admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.560221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 842/1672] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:41 2022] POST /identity/v3/users => generated 302 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:41.563818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85119663-a119-4e51-b091-79d591643aad admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.567792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-85119663-a119-4e51-b091-79d591643aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.597123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 831/1673] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:41.597123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.614002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85119663-a119-4e51-b091-79d591643aad admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.623978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3a747-42d6-4296-be9f-817c7fef05a7 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.626312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.629531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.629859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.630148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] PATH_INFO: `/v3/users/1ff7a96ff0cc4ead9fe25835d0546744` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.633531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff7a96ff0cc4ead9fe25835d0546744)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.635992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.636235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.636987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.639669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.640142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.641343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.647448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385342e5-3774-4999-9a12-5f289b1d8e41 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.660052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 832/1674] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:41.667464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.667945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.669057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.693990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] MFA Rules not processed for user `1ab4dbb6c98f4c99bbb06f586aaddeba`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:41.695463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.708870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 833/1675] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:41.708870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.732462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de9b14-eb2c-4693-bf2f-fff97dc5071e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.737878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.739222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.739494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.739900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.740699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.747167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.761102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 834/1676] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:41.761630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.767285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:41.898801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e59b5-5c1f-4667-9a89-69da44910508 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:41.900381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff7a96ff0cc4ead9fe25835d0546744'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:41.913369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948716fc-7c50-4473-9531-f424db23206f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:41.914862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948716fc-7c50-4473-9531-f424db23206f service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:41.915343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948716fc-7c50-4473-9531-f424db23206f service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:41.916474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948716fc-7c50-4473-9531-f424db23206f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:41.939747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948716fc-7c50-4473-9531-f424db23206f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:41.944664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948716fc-7c50-4473-9531-f424db23206f service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:41.968557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 835/1677] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:41.969851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948716fc-7c50-4473-9531-f424db23206f service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:41.990826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948716fc-7c50-4473-9531-f424db23206f service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:42.000732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:42.001875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:42.002390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:42.003512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] PATH_INFO: `/v3/projects/f64bceb5914c46ba96ed05750126f3a3/users/d018be0384014ec8a611955c2c0b311e/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:42.053080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff7a96ff0cc4ead9fe25835d0546744'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.054437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff7a96ff0cc4ead9fe25835d0546744'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.056323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff7a96ff0cc4ead9fe25835d0546744'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.070272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff7a96ff0cc4ead9fe25835d0546744'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.103413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64bceb5914c46ba96ed05750126f3a3, user_id=d018be0384014ec8a611955c2c0b311e, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:42.110699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49890fb5-5168-48ea-bb3b-6d23a62be903 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:42.144605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 843/1678] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:41 2022] DELETE /identity/v3/users/1ff7a96ff0cc4ead9fe25835d0546744 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:42.195466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:42.197386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:42.264041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 836/1679] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:41 2022] PUT /identity/v3/projects/f64bceb5914c46ba96ed05750126f3a3/users/d018be0384014ec8a611955c2c0b311e/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:42.270983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:42.271400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:42.271845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:42.342918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:42.348365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:42.407736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300137-7782-4ca9-b2d0-5d4e8c6bf8f1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:42.510114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 837/1680] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:42.596481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:42.597190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:42.597474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:42.597819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] PATH_INFO: `/v3/projects/f2d7c923153f4e01aceb606eca92ec8f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:42.599112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d7c923153f4e01aceb606eca92ec8f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:42.602591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:42.603928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:42.605466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:42.738797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade9dc3-31fb-4321-83b2-af65440af8ca None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:42.748110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:42.751276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:42.751572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:42.751887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:42.780937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:42.782490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e9254-b94b-4cb1-a9cb-6b8ece2c75b7 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:42.807023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 838/1681] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:26:42.818154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d7c923153f4e01aceb606eca92ec8f', 'request_id': 'req-b2e41656-23ed-4bef-80e6-b9001813276d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.840093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d7c923153f4e01aceb606eca92ec8f', 'request_id': 'req-b2e41656-23ed-4bef-80e6-b9001813276d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.973070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d7c923153f4e01aceb606eca92ec8f', 'request_id': 'req-b2e41656-23ed-4bef-80e6-b9001813276d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.992936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] The token cache is being invalidate because project f2d7c923153f4e01aceb606eca92ec8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:42.993222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:42.997076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b2e41656-23ed-4bef-80e6-b9001813276d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:43.005668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 844/1682] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:42 2022] DELETE /identity/v3/projects/f2d7c923153f4e01aceb606eca92ec8f => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:26:43.009042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.009337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.009667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.066732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:43.068000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:43.181882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:43.227791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 839/1683] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:43.230423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.231507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.231507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.269025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:43.270449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:43.287002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 845/1684] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:43.293256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:43.309334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e72cc-6a11-446d-bc76-ddd695de5988 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:43.321656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:43.322462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.322896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.323345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.324190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:43.324641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:43.326400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34cc11-4535-4058-bc94-828e296479d4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:43.351528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 840/1685] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:43.357616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:43.375088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23c20-19e9-4018-8a8d-80c6935ae983 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:43.380901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:43.381755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.382270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.382689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.383485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:43.385746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:43.457328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2f1a-1696-4c08-b7c7-78cc8915b212 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.457814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2f1a-1696-4c08-b7c7-78cc8915b212 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.458500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2f1a-1696-4c08-b7c7-78cc8915b212 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.461196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 841/1686] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 15 19:26:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:43.467812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.467812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.467812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.562477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 846/1687] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:43 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:43.563550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:43.573276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:43.575553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:43.580061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:43.697511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 842/1688] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:26:43.698835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:43.727429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8324bf3-a5d5-44bb-8b01-9a57476a1e6a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:43.746449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:43.747206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe75317-28c4-401f-b8db-4125400d4808 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:43.749281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.750030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.750964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] PATH_INFO: `/v3/projects/ecc8dce537dd4fb2901ca3b2b62a5dc2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.757629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc8dce537dd4fb2901ca3b2b62a5dc2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:43.763051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:43.764201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:43.764683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:43.766137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:43.768146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:43.768256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:43.775231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:43.915972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 843/1689] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:43 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:43.920667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:43.935549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:43.985812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d2ae7-9577-4d23-bbf2-0725e3913f26 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.003039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.003744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.004056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.004552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.029617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.044256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.049233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:44.056830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc8dce537dd4fb2901ca3b2b62a5dc2', 'request_id': 'req-e772482a-a0a9-412c-b04f-5203cc642f42'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.067302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 844/1690] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:44.068934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.070417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.073993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc8dce537dd4fb2901ca3b2b62a5dc2', 'request_id': 'req-e772482a-a0a9-412c-b04f-5203cc642f42'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.101865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3436-d703-4cd6-a3a6-ca105642ff5d service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.284167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.289426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.289928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.290388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] PATH_INFO: `/v3/projects/dd39254ccc1b454e82972a58b9c471bf` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.297330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd39254ccc1b454e82972a58b9c471bf)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.300419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.341001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc8dce537dd4fb2901ca3b2b62a5dc2', 'request_id': 'req-e772482a-a0a9-412c-b04f-5203cc642f42'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.364648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] The token cache is being invalidate because project ecc8dce537dd4fb2901ca3b2b62a5dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:44.365409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.368363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.370570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 847/1691] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:43 2022] DELETE /identity/v3/projects/ecc8dce537dd4fb2901ca3b2b62a5dc2 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:44.371552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.373137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.393274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772482a-a0a9-412c-b04f-5203cc642f42 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.405414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.406930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.407983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.408705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.411966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.419279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0b94-053f-48f1-97c3-f832074816a6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.436310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 848/1692] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:44.437986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.438716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.439591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.506044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:44.513336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.518854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd39254ccc1b454e82972a58b9c471bf', 'request_id': 'req-a57c9bc2-c431-459a-af1c-7a0086587575'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.532854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd39254ccc1b454e82972a58b9c471bf', 'request_id': 'req-a57c9bc2-c431-459a-af1c-7a0086587575'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.647931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 849/1693] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:44.648939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.653671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd39254ccc1b454e82972a58b9c471bf', 'request_id': 'req-a57c9bc2-c431-459a-af1c-7a0086587575'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.671297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] The token cache is being invalidate because project dd39254ccc1b454e82972a58b9c471bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:44.672192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.673803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a57c9bc2-c431-459a-af1c-7a0086587575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:44.677005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 845/1694] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:44 2022] DELETE /identity/v3/projects/dd39254ccc1b454e82972a58b9c471bf => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:44.686644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.687100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.687915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d61dde-24fb-46be-be42-1ddb9e675b41 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.688428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.705705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.707766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.707939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.710559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.711668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.716197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138bb52-c7d0-4b9b-bce1-4622db40d3bc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.744519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 850/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:44.745787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:44.747026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.748542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.748916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.749281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.771093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 846/1696] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:44.772708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.774380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.796677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:44.799208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.808619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f9d16-8784-4047-92c8-83008ef02082 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.829404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.841311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.841698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.842187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 851/1697] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:44.842574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] PATH_INFO: `/v3/projects/3a65ff57d5f24ac1be16cdd442c181bd/users/d62f1ef589be4c25b36a5d7c65539643/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.843601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.870729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9770b-2405-436e-8710-f078fdbf5298 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:44.883676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:44.884990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a65ff57d5f24ac1be16cdd442c181bd, user_id=d62f1ef589be4c25b36a5d7c65539643, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.885968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.887937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.888495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.889699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:44.890188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:44.891328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.893171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafd2e7-f6e4-47cf-84e1-37541666e94f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:44.911298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 852/1698] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:44.912923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:44.916789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:44.916789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:44.964868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:44.966366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:44.979762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 853/1699] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:44.979762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:44.989113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6b66d-b53c-49a3-bb12-a9a866135796 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.008486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.012261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.012775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.013103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.013865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.017087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08785d-cb95-4b6e-b646-587a5c9ce17a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.054319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 847/1700] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:44 2022] PUT /identity/v3/projects/3a65ff57d5f24ac1be16cdd442c181bd/users/d62f1ef589be4c25b36a5d7c65539643/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:45.056874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:45.180269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 854/1701] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:44 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:45.180269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.180269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.180269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.268834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] MFA Rules not processed for user `d62f1ef589be4c25b36a5d7c65539643`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:45.270210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:45.334116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bff52-765d-4357-8e02-412e1236f66b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.516665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 855/1702] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:45.516665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:45.517938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.518555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.520620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.520922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.522109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:26:45.522489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.528009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.539555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e09f-824d-47c3-a912-bfd706fba8d2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.551978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.552667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.552955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.553231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.554142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.562538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.573017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 848/1703] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 518 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:26:45.574071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:45.576366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:45.674154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 856/1704] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:45 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:45.676136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:45.682340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd339-f7ff-4679-8c2d-92ac49624c95 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.689375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.689650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444d943-6114-4fc2-ad46-9f77c8cc385f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.690018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.690357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.690760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.697751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:45.716998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.717905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.719784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.722779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.723618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.727280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.727280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.731586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da45b8-9b18-4629-aa0b-398082d7f93b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.737586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 849/1705] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:45.749521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:45.777859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abfc82-6217-49e4-90e1-d9a94c371638 service placement] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:45.790669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:45.791796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.792291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.793161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.796913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:45.800518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:45.844353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:45.942060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 857/1706] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:45 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:45.942060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:45.942060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:45.942060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:45.979720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 850/1707] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:45 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:45.985252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.043068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] MFA Rules not processed for user `0e37c8ca70ad4b6e8430a71d0f5e564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:46.046558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:46.166723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320af7c4-d814-4e0d-a7df-21a0d5b188c0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.179028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.180047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.180473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.181590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.184640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.192994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.215826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 858/1708] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:46.216620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.258993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3534c5e-8f80-4c3f-a44b-6b886977712a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.262554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 851/1709] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:46.264462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.291029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c9cb-b6d1-447c-b5ce-23cf6c63f574 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.295527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.296644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.297018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.298812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.299713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.300268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.300686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.301155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.301638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.302284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.304970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a70ae-1619-47fe-94e1-b1f7d09efc0c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.319447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.370669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:46.456297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 852/1710] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:46 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:46.458424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.461129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.461473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.554729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] MFA Rules not processed for user `d018be0384014ec8a611955c2c0b311e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:46.556025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:46.557976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 859/1711] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:46 2022] POST /identity/v3/users => generated 297 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:46.559611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.565337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:46.710832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 853/1712] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:46.711970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.735514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ac50d-3ab4-4b80-ab6a-cd9d7bb30378 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.741187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641bc4-76f2-4313-a120-873253e37576 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.747685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.747841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.748389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.748745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.749138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.749469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.750074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.750529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.751827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.754499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.764561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.766470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec90e9-53b5-4a31-a605-469070b7030d service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.777636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 860/1713] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:46.779098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.779397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.779704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.783136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 854/1714] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:46.783136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.802497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bdff-21be-4ee7-bd7f-5ff33759cc92 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.819820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.819820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.819820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.819820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.824234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.851444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:46.862013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] MFA Rules not processed for user `1d7c40b5232e46119434aeeb8a6ad4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:46.864562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:46.890884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 855/1715] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:46.890884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:46.955497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073951f-e30f-49d6-8801-0eb35b647a6e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:46.972594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:46.980017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:46.980443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:46.981077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:46.982052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:46.983913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:47.007772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 856/1716] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:47.008898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:47.020181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:47.066449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3407124-77e3-4593-8a83-e3d9d5b74ade admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:47.087224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:47.088190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:47.088682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:47.089224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:47.090102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:47.092547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:47.167050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 861/1717] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:47.168182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:47.184106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbdccf-e086-46f2-8235-4e6727962ed5 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:47.197765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:47.198530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:47.198894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:47.199201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] PATH_INFO: `/v3/projects/c1da2772aedb41dc979be7e8030b7bd8/users/8a20e9d8329946ab82c29a17574b4f5f/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:47.222605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1da2772aedb41dc979be7e8030b7bd8, user_id=8a20e9d8329946ab82c29a17574b4f5f, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:47.230364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:47.333137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 857/1718] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:47 2022] POST /identity/v3/projects => generated 405 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:47.334177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:47.388560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 862/1719] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:47 2022] PUT /identity/v3/projects/c1da2772aedb41dc979be7e8030b7bd8/users/8a20e9d8329946ab82c29a17574b4f5f/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:47.390128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:47.535686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8338af-2529-47d7-a40c-da950f2b1b55 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:47.565074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b3618-bef5-47ac-953a-39845ef0a6e4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:47.574069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:47.580030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:47.580768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:47.581050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:47.585502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] PATH_INFO: `/v3/projects/95d03a896eb44a1da708252ba32158c8/users/5b87abc01a0c4169b35025dca68206e2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:47.620665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d03a896eb44a1da708252ba32158c8, user_id=5b87abc01a0c4169b35025dca68206e2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:47.626986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7944fbb-2ea1-4370-98d0-203decba674a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:47.711232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 863/1720] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:47 2022] PUT /identity/v3/projects/95d03a896eb44a1da708252ba32158c8/users/5b87abc01a0c4169b35025dca68206e2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:47.713100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258996ae-be8b-421c-9e80-ecfa8d74c6fd None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:47.716456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258996ae-be8b-421c-9e80-ecfa8d74c6fd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:47.717017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258996ae-be8b-421c-9e80-ecfa8d74c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:47.734334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:47.735493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:47.739128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:47.739128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:47.744520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:47.761273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:47.796791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 858/1721] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:47.799000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:47.811936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-258996ae-be8b-421c-9e80-ecfa8d74c6fd None None] MFA Rules not processed for user `1d7c40b5232e46119434aeeb8a6ad4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:47.820162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-258996ae-be8b-421c-9e80-ecfa8d74c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:47.841897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064f8c-379b-45cc-9c2d-c62ebbefb748 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.018929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.019743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.020108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.020500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.021269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.024009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.037971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 864/1722] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:48.038518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.038628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.040189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.078230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] MFA Rules not processed for user `8a20e9d8329946ab82c29a17574b4f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:48.079625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:48.144171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 859/1723] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:47 2022] POST /identity/v3/users => generated 309 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:48.145364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.181677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e64ba-bf80-4c54-a61d-1551b0888c0c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.213014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.214281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.215566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 865/1724] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:48.216970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.217523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.217935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.219314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.226869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.239112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc7d0c-ffb3-4c08-ae39-e4c978e3b6fb None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.248977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.251942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.251942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.251942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] PATH_INFO: `/v3/projects/28c1a9dd228b4a70b67da7618faf52e5/users/42ba8ea3d064459e8688978d1d3cd426/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.273310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 860/1725] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:48.276161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c1a9dd228b4a70b67da7618faf52e5, user_id=42ba8ea3d064459e8688978d1d3cd426, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.279632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.290210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.457759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 866/1726] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:48 2022] PUT /identity/v3/projects/28c1a9dd228b4a70b67da7618faf52e5/users/42ba8ea3d064459e8688978d1d3cd426/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:48.457759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.487718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84764-ce82-47e9-b1e0-98f4ad24e562 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.674471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d06c8-0e7a-449e-a282-203d32388c1b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.695608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.696428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.696792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.697042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.697731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.699515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.721086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.722461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.722927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.723381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.731375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 867/1727] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:48.733728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.746782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132964-0df8-4342-bdab-0f3b398fed6a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.753161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.754586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.756580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.756869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.757546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.759514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.811097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.812762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.823830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 861/1728] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:48.825307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:48.837792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e4752-098d-409c-b309-ebb5b7c572e6 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:48.842981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:48.843736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.844130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.844419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] PATH_INFO: `/v3/projects/95d03a896eb44a1da708252ba32158c8/users/5b87abc01a0c4169b35025dca68206e2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.846395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d03a896eb44a1da708252ba32158c8, user_id=5b87abc01a0c4169b35025dca68206e2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:48.849462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39b7d-af7b-4200-8282-d3f5240aee4e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:48.865694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 862/1729] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:48 2022] PUT /identity/v3/projects/95d03a896eb44a1da708252ba32158c8/users/5b87abc01a0c4169b35025dca68206e2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:48.867019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b72c-d5a1-4973-a7bf-8dd09c9b0d88 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:48.867308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b72c-d5a1-4973-a7bf-8dd09c9b0d88 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:48.867587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b72c-d5a1-4973-a7bf-8dd09c9b0d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:48.957858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-5f75b72c-d5a1-4973-a7bf-8dd09c9b0d88 None None] MFA Rules not processed for user `42ba8ea3d064459e8688978d1d3cd426`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:48.960430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5f75b72c-d5a1-4973-a7bf-8dd09c9b0d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:48.996562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 868/1730] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:48 2022] POST /identity/v3/projects => generated 371 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:49.000371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:49.183046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:49.199258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 863/1731] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:49.199258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:49.199258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:49.199258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:49.244149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c1e7-a8ee-427a-a88e-3694b96ad6db admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:49.268042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:49.269736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:49.276025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:49.276740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] PATH_INFO: `/v3/projects/8531d37ffae34055887febfdd231f33a/users/d66e56cc572b487a81912f2c02d8a67e/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:49.320766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] MFA Rules not processed for user `5b87abc01a0c4169b35025dca68206e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:49.326799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:49.328276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8531d37ffae34055887febfdd231f33a, user_id=d66e56cc572b487a81912f2c02d8a67e, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:49.336354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:49.473567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:49.487040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 869/1732] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:48 2022] PUT /identity/v3/projects/8531d37ffae34055887febfdd231f33a/users/d66e56cc572b487a81912f2c02d8a67e/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:49.491776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:49.630380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 864/1733] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:49.632507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:49.671224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc327c-dc13-4b54-806c-f55f70180a8b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:49.735388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a509e-3811-419c-9cb6-67655cb43bc7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:49.760426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:49.762771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:49.763252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:49.763749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:49.768220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:49.780972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:49.910031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:49.910674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:49.910960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:49.911230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:49.914328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 870/1734] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:49 2022] POST /identity/v3/users => generated 292 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:49.916345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:49.920615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:49.923280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:49.950778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5077845-36b5-4d0b-875b-f2a1dab9859e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:49.970195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:49.970856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:49.971139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:49.971416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:49.972111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:49.974191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:49.990732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 871/1735] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:49.990732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:50.001514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:50.113364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 865/1736] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:26:49 2022] POST /identity/v3/projects => generated 377 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:50.114796 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:50.154069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a382f-e7a6-4252-9135-4b27cee1838e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:50.330188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:50.331103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:50.331572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:50.332071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:50.397281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225688f1-5445-4984-a383-7439041b98a7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:50.430813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:50.432470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:50.432962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:50.433385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:50.436415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:50.442549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:50.468971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 866/1737] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:50.470707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:50.487792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6801554-3d33-4d17-9f76-02b8992be2d7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:50.498031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:50.498203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:50.499159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:50.499664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:50.499772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca723ab9-d043-4d7a-b084-225b6f54d52b service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:50.500166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] PATH_INFO: `/v3/projects/8531d37ffae34055887febfdd231f33a/users/d66e56cc572b487a81912f2c02d8a67e/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:50.503269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8531d37ffae34055887febfdd231f33a, user_id=d66e56cc572b487a81912f2c02d8a67e, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:50.510443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:50.531412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 867/1738] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:50 2022] PUT /identity/v3/projects/8531d37ffae34055887febfdd231f33a/users/d66e56cc572b487a81912f2c02d8a67e/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:50.532395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 872/1739] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:50.533410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:50.533906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:50.534219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:50.551830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:50.604880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] MFA Rules not processed for user `1d7c40b5232e46119434aeeb8a6ad4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:50.606333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:50.811103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 873/1740] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:50.813907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:50.894367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3fc70-eedb-44bd-b9db-fe46d32b4f50 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:50.923411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:50.924096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:50.928466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:50.928466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:50.928466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:50.928466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.065812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496640e7-e12e-417f-8871-2c6ee830681a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.082639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.083409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.083610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.084469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] PATH_INFO: `/v3/projects/618228ae110a4169b0523acdc864665f/users/afb2ed844e9c432187e22927df8e0b3f/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.116230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 868/1741] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:26:50 2022] POST /identity/v3/users => generated 295 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:26:51.116230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:51.119910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=618228ae110a4169b0523acdc864665f, user_id=afb2ed844e9c432187e22927df8e0b3f, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.122834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.268795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 874/1742] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:50 2022] PUT /identity/v3/projects/618228ae110a4169b0523acdc864665f/users/afb2ed844e9c432187e22927df8e0b3f/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:51.270008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:51.350760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a44c1-9b98-4cd6-ada2-4636c9aacd42 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.374433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:51.485978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9740a-bd4b-4a07-a687-9016101c4d9d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.507089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.507958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.509886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.510285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.513512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.517693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.539435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.557530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 875/1743] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:51.558756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:51.559001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.559529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.561667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.611944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390576b-a3d1-4890-965c-9642827f6912 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.625150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.628856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.629412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.629886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.631519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.635136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.664655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 876/1744] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:51.667519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:51.688447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5fa1-8ab2-4a43-9e7b-122329bca3fa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.714891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.715613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.715964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.716195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] PATH_INFO: `/v3/projects/b9802105d7464fac9969f3f68c32f0da/users/b37a5acad6c94ce99a504fee59e43324/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.749860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.752410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.765107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 869/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:51.766082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:51.779025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224cb75-10af-45b7-bcdf-bf0013008b21 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:51.783438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9802105d7464fac9969f3f68c32f0da, user_id=b37a5acad6c94ce99a504fee59e43324, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.784452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:51.785234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.786227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.786227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] PATH_INFO: `/v3/projects/618228ae110a4169b0523acdc864665f/users/afb2ed844e9c432187e22927df8e0b3f/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.788646 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618228ae110a4169b0523acdc864665f, user_id=afb2ed844e9c432187e22927df8e0b3f, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:51.791802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf601-385a-48da-8ab0-1e264622b970 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.793674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0aa80-3909-49c5-8632-14d6ea19a09f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:51.805953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 870/1746] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:51 2022] PUT /identity/v3/projects/618228ae110a4169b0523acdc864665f/users/afb2ed844e9c432187e22927df8e0b3f/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:51.821482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.821956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.823184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:51.843075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:51.865057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] MFA Rules not processed for user `afb2ed844e9c432187e22927df8e0b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:51.868070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:51.900081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 877/1747] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:51 2022] PUT /identity/v3/projects/b9802105d7464fac9969f3f68c32f0da/users/b37a5acad6c94ce99a504fee59e43324/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:51.919032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53347775-8836-445c-8549-0860c25323d8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:51.919354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53347775-8836-445c-8549-0860c25323d8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:51.924429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53347775-8836-445c-8549-0860c25323d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.115847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-53347775-8836-445c-8549-0860c25323d8 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:52.117126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-53347775-8836-445c-8549-0860c25323d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:52.171004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 871/1748] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:52.172643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:52.285068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 878/1749] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:52.285964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53347775-8836-445c-8549-0860c25323d8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:52.288276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-53347775-8836-445c-8549-0860c25323d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:52.385797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b246-f81c-4c2f-b1ac-4245144de233 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:52.488680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53347775-8836-445c-8549-0860c25323d8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:52.511941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:52.512598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:52.512872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:52.513168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] PATH_INFO: `/v3/users/07689aa819d8466394c59d8044543c30` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.514484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] RBAC: Authorizing `identity:delete_user(user_id=07689aa819d8466394c59d8044543c30)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:52.516466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:52.587715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:52.589787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:52.590274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:52.590765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.592675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:52.595202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:52.623324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 872/1750] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:26:52.623324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:52.623324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:52.743032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572ceff-b14a-4d3a-9733-aa77810cd022 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:52.747181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:52.753254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:52.753254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:52.753254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.873961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07689aa819d8466394c59d8044543c30'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:52.879476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:52.881866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:52.892955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 873/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:52.893881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:52.906472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524a2d-6b97-4576-a080-2287e8c8e904 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:52.916456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:52.917149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:52.917442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:52.917726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] PATH_INFO: `/v3/projects/b9802105d7464fac9969f3f68c32f0da/users/b37a5acad6c94ce99a504fee59e43324/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.919856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9802105d7464fac9969f3f68c32f0da, user_id=b37a5acad6c94ce99a504fee59e43324, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:52.927071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905b2e8-2b4a-4111-8db1-55d4ba489bb3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:52.939056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 874/1752] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:26:52 2022] PUT /identity/v3/projects/b9802105d7464fac9969f3f68c32f0da/users/b37a5acad6c94ce99a504fee59e43324/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:52.940851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:52.941228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:52.941611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:52.944468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 875/1753] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 15 19:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:52.945447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:52.947269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.026061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07689aa819d8466394c59d8044543c30'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:53.027998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07689aa819d8466394c59d8044543c30'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:53.029509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07689aa819d8466394c59d8044543c30'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:53.089725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2f019a18-71c2-4aa3-97b4-edd21437536d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07689aa819d8466394c59d8044543c30'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:53.137806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 879/1754] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:52 2022] DELETE /identity/v3/users/07689aa819d8466394c59d8044543c30 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:53.139564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5e3-3a30-4bfd-81d8-291707d65124 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.140395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5e3-3a30-4bfd-81d8-291707d65124 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.140825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5e3-3a30-4bfd-81d8-291707d65124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.172357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac625b21-3448-466c-b4e4-481ce0188f1d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:53.198693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-eb48d5e3-3a30-4bfd-81d8-291707d65124 None None] MFA Rules not processed for user `b37a5acad6c94ce99a504fee59e43324`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:53.202503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-eb48d5e3-3a30-4bfd-81d8-291707d65124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.308713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:53.309371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.309815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.310441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.312899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.388107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 880/1755] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:53.391304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.405354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.408274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.454814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:53.456575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269979a2-c9a1-42d4-b7cc-6d9c10aa1946 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:53.458134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:53.462471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.487450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 876/1756] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:26:53.491370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14283-71f5-46fe-94ee-4de852722018 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.492012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14283-71f5-46fe-94ee-4de852722018 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.493377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14283-71f5-46fe-94ee-4de852722018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.546579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8fc14283-71f5-46fe-94ee-4de852722018 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:53.559549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8fc14283-71f5-46fe-94ee-4de852722018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.646891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 881/1757] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:53.648267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:53.650288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.774491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 877/1758] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:53.776801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.777239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.777650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.820412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff328d-d4e5-4f85-87d1-545c77fbb2ef None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:53.854370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:53.855010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:53.855295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:53.857018 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:53.859367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:53.864553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] MFA Rules not processed for user `d66e56cc572b487a81912f2c02d8a67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:53.866770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:54.031454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:54.033410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:54.044793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 878/1759] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:54.061466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:54.063511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:54.067688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 882/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:54.071472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:54.077987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:54.084097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff3ecb-d6ae-460c-bd9d-aa6ffe58c686 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:54.090752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:54.107388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:54.108304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:54.109181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:54.117144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182dfd1-ff63-43cc-91c2-2f0aa4f0e972 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:54.126876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:54.128071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:54.128561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:54.131053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:54.137973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:54.143789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7f209-5882-4350-8ffe-4f2b6a5ae3ad service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:54.159382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:54.160702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:54.167173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:54.176758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 879/1761] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:54.190010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:54.190622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:54.191097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:54.209969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 883/1762] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:54.260053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] MFA Rules not processed for user `0e37c8ca70ad4b6e8430a71d0f5e564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:54.263238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:54.306389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:54.486169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 880/1763] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:55.586557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:56.322366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:56.325017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:56.359529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18450d0a-05fe-466d-9a88-53b9cbe0e205 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:56.376675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:56.381301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:56.381301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:56.381301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] PATH_INFO: `/v3/users/8a20e9d8329946ab82c29a17574b4f5f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:56.386740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a20e9d8329946ab82c29a17574b4f5f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:56.394388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:56.437647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:56.798613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a20e9d8329946ab82c29a17574b4f5f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:56.931279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a20e9d8329946ab82c29a17574b4f5f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:56.932191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a20e9d8329946ab82c29a17574b4f5f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:56.936003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a20e9d8329946ab82c29a17574b4f5f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:56.965168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a9c32dd-2f19-43b7-a377-a35dec3350e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a20e9d8329946ab82c29a17574b4f5f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.002789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:57.006405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:57.027430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 884/1764] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:56 2022] DELETE /identity/v3/users/8a20e9d8329946ab82c29a17574b4f5f => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:57.037639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80807cfe-001a-4ffa-888f-8216a2cf6685 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:57.052161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:57.053012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:57.053300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:57.093637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:57.096120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:57.174713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:57.222562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:57.223363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:57.223760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:57.224037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] PATH_INFO: `/v3/projects/73be7c05690c456cbee1481590453dd5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:57.226110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73be7c05690c456cbee1481590453dd5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:57.228341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:57.412672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 885/1765] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:57.473443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73be7c05690c456cbee1481590453dd5', 'request_id': 'req-469527fc-4d37-46af-b731-8ca49e7b8c64'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.523257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73be7c05690c456cbee1481590453dd5', 'request_id': 'req-469527fc-4d37-46af-b731-8ca49e7b8c64'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.657273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73be7c05690c456cbee1481590453dd5', 'request_id': 'req-469527fc-4d37-46af-b731-8ca49e7b8c64'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.678390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] The token cache is being invalidate because project 73be7c05690c456cbee1481590453dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:57.678687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.680206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:57.682954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 881/1766] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:56 2022] DELETE /identity/v3/projects/73be7c05690c456cbee1481590453dd5 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:57.686721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:57.688471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:57.704720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:57.706268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:57.742285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d42a6f-4acd-4d21-9edb-aa45afc4b0f8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:57.807757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469527fc-4d37-46af-b731-8ca49e7b8c64 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:57.819982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:57.820625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:57.820914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:57.821203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:57.843958 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:57.938178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:57.939069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:57.939642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:57.940150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] PATH_INFO: `/v3/users/b8cc749441a14eafbe31aa106a326b60` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:57.941635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cc749441a14eafbe31aa106a326b60)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:57.944252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:58.101381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:58.110247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b37e07-53b9-4c61-aab8-e86634a3d7c7 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:58.152231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 882/1767] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:58.153587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a16e-47ff-4694-b597-cd2dfccca5d1 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.153868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a16e-47ff-4694-b597-cd2dfccca5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.154155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a16e-47ff-4694-b597-cd2dfccca5d1 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.168127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 883/1768] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:26:58 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:58.169880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd4b1-76fe-4cc7-8bb3-67774ab75615 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.170457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd4b1-76fe-4cc7-8bb3-67774ab75615 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.170805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd4b1-76fe-4cc7-8bb3-67774ab75615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.212621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0ebdd4b1-76fe-4cc7-8bb3-67774ab75615 None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:58.214135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0ebdd4b1-76fe-4cc7-8bb3-67774ab75615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:58.228570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cc749441a14eafbe31aa106a326b60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:58.234051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 884/1769] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 15 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:58.281687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dfe99-9e8a-4847-b8e5-a8f9de7de38f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.282260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dfe99-9e8a-4847-b8e5-a8f9de7de38f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.285636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dfe99-9e8a-4847-b8e5-a8f9de7de38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.343445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3c5dfe99-9e8a-4847-b8e5-a8f9de7de38f None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:58.343445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3c5dfe99-9e8a-4847-b8e5-a8f9de7de38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:58.369003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cc749441a14eafbe31aa106a326b60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:58.370024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cc749441a14eafbe31aa106a326b60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:58.372076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cc749441a14eafbe31aa106a326b60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:58.392344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e8f55674-5d94-474d-8bed-aad193444415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cc749441a14eafbe31aa106a326b60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:58.447388 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 886/1770] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:57 2022] DELETE /identity/v3/users/b8cc749441a14eafbe31aa106a326b60 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:58.454846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:58.490932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 885/1771] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:26:58.529722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733850-b4d0-47d1-a5ad-1cfb39e7bfcc None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.530342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733850-b4d0-47d1-a5ad-1cfb39e7bfcc None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.530825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733850-b4d0-47d1-a5ad-1cfb39e7bfcc None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.532580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 887/1772] 10.222.0.18 () {54 vars in 961 bytes} [Thu Sep 15 19:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:26:58.547973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.548589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.549157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.584065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:26:58.586117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:58.717039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 886/1773] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Sep 15 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:26:58.726321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e733850-b4d0-47d1-a5ad-1cfb39e7bfcc None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:58.752307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e733850-b4d0-47d1-a5ad-1cfb39e7bfcc None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:58.759150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:58.760558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:58.760920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:58.761324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:58.775487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:58.777389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:58.811754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 888/1774] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:58.955224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:59.248529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:59.250005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:59.331383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:59.333126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:59.513671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaadad-de20-451a-9131-e62618258ee6 service placement] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:59.521040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:59.522258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:59.523176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:59.523837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:59.526286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:26:59.545446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:59.548333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:59.567448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 889/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:26:59.592285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6c87d-1db2-4804-88d4-68418af5550d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:26:59.624752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:26:59.625129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:26:59.625311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:26:59.625592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] PATH_INFO: `/v3/projects/c1da2772aedb41dc979be7e8030b7bd8` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:26:59.630973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1da2772aedb41dc979be7e8030b7bd8)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:26:59.634706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:26:59.799203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:26:59.810863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1da2772aedb41dc979be7e8030b7bd8', 'request_id': 'req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:59.818600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1da2772aedb41dc979be7e8030b7bd8', 'request_id': 'req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:59.935770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1da2772aedb41dc979be7e8030b7bd8', 'request_id': 'req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:59.960722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] The token cache is being invalidate because project c1da2772aedb41dc979be7e8030b7bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:26:59.961114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:59.961914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4a3067a9-3427-4da5-93cb-ef3b42bb05ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:26:59.967212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 887/1776] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:26:59 2022] DELETE /identity/v3/projects/c1da2772aedb41dc979be7e8030b7bd8 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:26:59.968417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:26:59.982615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838894a-91d9-4071-ae8c-224eeb28936c service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:00.014814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9f347-bfd7-4297-909d-e38663ecad32 None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.015247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9f347-bfd7-4297-909d-e38663ecad32 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.016058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9f347-bfd7-4297-909d-e38663ecad32 None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.022493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 888/1777] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:27:00.030553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.030553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.030553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.083742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] MFA Rules not processed for user `bc38e83dcf4b4cd794bbc5c550d7fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:00.083742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.083742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:00.139695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:00.146407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.146407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.146407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] PATH_INFO: `/v3/users/afb2ed844e9c432187e22927df8e0b3f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.146407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb2ed844e9c432187e22927df8e0b3f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:00.149636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:00.238196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 889/1778] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:27:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:00.239155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:00.241232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.264416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fca5f8-f25d-4b1e-91d8-d6b988324985 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:00.274808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:00.276662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.277542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.278142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] PATH_INFO: `/v3/projects/6d4cda75c68d400f817239f82595f869` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.282529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4cda75c68d400f817239f82595f869)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:00.286719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:00.412942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb2ed844e9c432187e22927df8e0b3f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.452677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4cda75c68d400f817239f82595f869', 'request_id': 'req-40922052-f42f-437e-8bd5-1fd383346fd9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.462607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4cda75c68d400f817239f82595f869', 'request_id': 'req-40922052-f42f-437e-8bd5-1fd383346fd9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.526355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb2ed844e9c432187e22927df8e0b3f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.526981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb2ed844e9c432187e22927df8e0b3f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.527602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb2ed844e9c432187e22927df8e0b3f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.536337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb2ed844e9c432187e22927df8e0b3f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.558572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 890/1779] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:26:59 2022] DELETE /identity/v3/users/afb2ed844e9c432187e22927df8e0b3f => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:00.560829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:00.563025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.597413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4cda75c68d400f817239f82595f869', 'request_id': 'req-40922052-f42f-437e-8bd5-1fd383346fd9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.626293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] The token cache is being invalidate because project 6d4cda75c68d400f817239f82595f869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:00.626293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.628526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:00.634147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 890/1780] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:00 2022] DELETE /identity/v3/projects/6d4cda75c68d400f817239f82595f869 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:00.634147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:00.634147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.694277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa83b5-386c-4895-9ddb-850e9298c16b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:00.701868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:00.702805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.703200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.703677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.705737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.744556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40922052-f42f-437e-8bd5-1fd383346fd9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:00.747413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:00.750528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d7d7e-c6e9-4a2b-bfab-cedbc0bfa06f service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:00.765191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 891/1781] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:00.782535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bac69-27c4-4b6e-977c-4ed1bc2f8867 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.782920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bac69-27c4-4b6e-977c-4ed1bc2f8867 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.785465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bac69-27c4-4b6e-977c-4ed1bc2f8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.834810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:00.858925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ff7bac69-27c4-4b6e-977c-4ed1bc2f8867 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:00.860220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ff7bac69-27c4-4b6e-977c-4ed1bc2f8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:00.910382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:00.910382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:00.910382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:00.910382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] PATH_INFO: `/v3/users/d018be0384014ec8a611955c2c0b311e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:00.911574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d018be0384014ec8a611955c2c0b311e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:00.920610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:01.016697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 892/1782] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:01.020683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7673-76ed-4bcd-acb3-8df30e59c33e None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.021008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7673-76ed-4bcd-acb3-8df30e59c33e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.021472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7673-76ed-4bcd-acb3-8df30e59c33e None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.025116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 893/1783] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:27:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:27:01.029367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3b5e-27a3-4fac-9060-198670bd7857 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.029698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3b5e-27a3-4fac-9060-198670bd7857 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.030801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3b5e-27a3-4fac-9060-198670bd7857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.099921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d56f3b5e-27a3-4fac-9060-198670bd7857 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:01.101193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:01.103213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d56f3b5e-27a3-4fac-9060-198670bd7857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:01.137475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 894/1784] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:01.141566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.143541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.144332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.210741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:01.212042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:01.278360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd018be0384014ec8a611955c2c0b311e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.415902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd018be0384014ec8a611955c2c0b311e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.435844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd018be0384014ec8a611955c2c0b311e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.435844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd018be0384014ec8a611955c2c0b311e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.464574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 895/1785] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:01.465776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:01.468835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:01.481566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd018be0384014ec8a611955c2c0b311e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.523535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85317ed-78a8-420c-9998-69bb1e831673 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:01.533098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:01.534659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.535163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.535812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.537587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:01.539757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:01.540242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 891/1786] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:00 2022] DELETE /identity/v3/users/d018be0384014ec8a611955c2c0b311e => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:01.543624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:01.551601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:01.551995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:01.572284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 896/1787] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:01.573815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:01.576805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:01.607790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826a051-fc69-4232-8586-710164befc00 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:01.715533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba297b-d4ac-4eb1-8398-746c831f46bf admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:01.729152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:01.732331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.733283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.734560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] PATH_INFO: `/v3/projects/618228ae110a4169b0523acdc864665f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.736510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618228ae110a4169b0523acdc864665f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:01.739257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:01.832412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:01.836673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.836673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.836673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.850002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:01.852101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:01.868654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 897/1788] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:01.869433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:01.885862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a787729-9ec7-4a6f-852a-c78359e70d84 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:01.892698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:01.893483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:01.894016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:01.894397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:01.895164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:01.898804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:01.922603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618228ae110a4169b0523acdc864665f', 'request_id': 'req-39536e7a-b4b3-4ec2-8664-b522a82f8303'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:01.936675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618228ae110a4169b0523acdc864665f', 'request_id': 'req-39536e7a-b4b3-4ec2-8664-b522a82f8303'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.023396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618228ae110a4169b0523acdc864665f', 'request_id': 'req-39536e7a-b4b3-4ec2-8664-b522a82f8303'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.027451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:02.041267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] The token cache is being invalidate because project 618228ae110a4169b0523acdc864665f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:02.041638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.042488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.044827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 892/1789] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:01 2022] DELETE /identity/v3/projects/618228ae110a4169b0523acdc864665f => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:02.046302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 898/1790] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:01 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:02.047937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.049219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.050074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.050922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.156525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71701f07-7989-4872-9b19-8ffa432ff22d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.157639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39536e7a-b4b3-4ec2-8664-b522a82f8303 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.164735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.165681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.165888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.166742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.166925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.167191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.167377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.167643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] PATH_INFO: `/v3/projects/f64bceb5914c46ba96ed05750126f3a3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.168483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.169302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64bceb5914c46ba96ed05750126f3a3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.171550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.172915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed0f2-e17a-4792-843a-3a2f90818fe6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.216717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:02.299172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 899/1791] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:02 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:02.299172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.299172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.299172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.331251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:02.333118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.351958 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64bceb5914c46ba96ed05750126f3a3', 'request_id': 'req-51b20f75-6211-48b3-8171-f1e8fe8df317'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.367904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64bceb5914c46ba96ed05750126f3a3', 'request_id': 'req-51b20f75-6211-48b3-8171-f1e8fe8df317'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.441912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 900/1792] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:02.443871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.453390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64bceb5914c46ba96ed05750126f3a3', 'request_id': 'req-51b20f75-6211-48b3-8171-f1e8fe8df317'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.459356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24cf15-82d0-4d3d-bed7-8732d84ea7de None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.465892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.466924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.467371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.467792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.468565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.470596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.472242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] The token cache is being invalidate because project f64bceb5914c46ba96ed05750126f3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:02.472762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.473658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51b20f75-6211-48b3-8171-f1e8fe8df317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.475836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 893/1793] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:02 2022] DELETE /identity/v3/projects/f64bceb5914c46ba96ed05750126f3a3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:02.477272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.477707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.478328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.488870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 901/1794] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:02.490584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.492412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.499762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:02.500976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.506980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09b29-6076-44a1-84cb-4e852524bdfa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.509417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 894/1795] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:02.510239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.511548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.520410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.521575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.521851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.522130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] PATH_INFO: `/v3/users/d66e56cc572b487a81912f2c02d8a67e` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.523345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66e56cc572b487a81912f2c02d8a67e)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.525340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.529550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe8cda-af9a-4ab9-a933-a137fa27efb2 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.536237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.537925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.538412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.538896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.540774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.543170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.566752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 895/1796] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:02.566752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.584584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5da7-6bf4-4726-b907-fbefbb58923a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.592569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.593613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.594527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.594644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.595829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:02.596371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.600856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.630045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 896/1797] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:02.630045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.648465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4de42-a4f9-4dd0-82e7-feaaee0031df admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.663102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:02.665086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.665086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.665086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] PATH_INFO: `/v3/projects/4b1d52015b8a4bdaac64510b7d151ec7/users/416cbb72fc6848edb1baafafec514d91/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.697938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1d52015b8a4bdaac64510b7d151ec7, user_id=416cbb72fc6848edb1baafafec514d91, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:02.704819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:02.724164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:02.749021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66e56cc572b487a81912f2c02d8a67e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.826169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66e56cc572b487a81912f2c02d8a67e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.827513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66e56cc572b487a81912f2c02d8a67e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.828602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66e56cc572b487a81912f2c02d8a67e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.836934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 897/1798] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:02 2022] PUT /identity/v3/projects/4b1d52015b8a4bdaac64510b7d151ec7/users/416cbb72fc6848edb1baafafec514d91/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:02.838628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:02.847143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:02.852947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d0e39abd-f32e-4632-a4b5-4b208d8b81f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66e56cc572b487a81912f2c02d8a67e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:02.908374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 902/1799] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:02 2022] DELETE /identity/v3/users/d66e56cc572b487a81912f2c02d8a67e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:02.919786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:02.920256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:02.920715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:02.929775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:02.964761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6a0f-02f2-412c-8d7f-0014353c27d2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:02.984647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] MFA Rules not processed for user `416cbb72fc6848edb1baafafec514d91`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:02.986183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.053234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.054123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.054408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.054878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.055516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.057572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.101129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 903/1800] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:03.105995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.107553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.123492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50913a-8297-442d-bb02-bfcaac424588 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.128695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.129312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.129597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.129871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.130848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.132567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.170200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 898/1801] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:02 2022] POST /identity/v3/projects => generated 363 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:03.172901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.245595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4298-b1b2-41ab-b372-768864796ea4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.252680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.256502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.256502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.256502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.256502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.259314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.263594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 904/1802] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:03.264610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.346100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9981e-03d0-48f6-a77f-70818f128826 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.351445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 899/1803] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:03.352907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.354650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.355329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.355510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.355606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.355910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.356579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.358519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.377545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a406e-14fd-4340-90ee-baf46aba7a4b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.385030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.385906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.386251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.386639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] PATH_INFO: `/v3/projects/8531d37ffae34055887febfdd231f33a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.387826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8531d37ffae34055887febfdd231f33a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.389708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.420378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:03.480890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 905/1804] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:03.480890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.488717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311885-f98d-490d-b694-223a86c4ad4f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.495635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.497845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.497845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.497845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.497845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.498987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.509461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 906/1805] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:03.510306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.517625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8531d37ffae34055887febfdd231f33a', 'request_id': 'req-b251bf88-9c89-4e1a-86fc-1c5c67e34366'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:03.520416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43602-0fec-4c79-8f97-33c101c7e251 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.527359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8531d37ffae34055887febfdd231f33a', 'request_id': 'req-b251bf88-9c89-4e1a-86fc-1c5c67e34366'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:03.582745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.583357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.584213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.584619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.585259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.586853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.596671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 907/1806] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:03.597601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.600565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8531d37ffae34055887febfdd231f33a', 'request_id': 'req-b251bf88-9c89-4e1a-86fc-1c5c67e34366'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:03.607709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922946-b418-4b77-843e-d7e8b4793635 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.613039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.613636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.613921 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.614201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.614860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.615135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] The token cache is being invalidate because project 8531d37ffae34055887febfdd231f33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:03.615639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:03.616488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1ebe9-80c7-46f6-b70e-5cf7555081f0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.616717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:03.618625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 900/1807] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:03 2022] DELETE /identity/v3/projects/8531d37ffae34055887febfdd231f33a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:03.619582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.621092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.636327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 908/1808] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:03.636327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c218351-731f-4c92-876e-0e980114407a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.636327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c218351-731f-4c92-876e-0e980114407a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.636327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c218351-731f-4c92-876e-0e980114407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.638484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bf88-9c89-4e1a-86fc-1c5c67e34366 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.644188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.644949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.645323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.645786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] PATH_INFO: `/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a/users/e810aced3d47464caf483976573cf3e9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.654596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0c218351-731f-4c92-876e-0e980114407a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:03.655905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0c218351-731f-4c92-876e-0e980114407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.663956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 909/1809] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:03.664698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c218351-731f-4c92-876e-0e980114407a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.665772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d542f66f40d4a1bb39a8f26f4563a0a, user_id=e810aced3d47464caf483976573cf3e9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.665957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0c218351-731f-4c92-876e-0e980114407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.669285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cca85-59f4-47e9-ba66-027b2eef15c3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.678516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c218351-731f-4c92-876e-0e980114407a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.683422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.684122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.684418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.692403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] PATH_INFO: `/v3/projects/84e0c1e8dbd04905b0d1963f4f00b275/users/18a6f40d6f9d48ebb71840e8abc0bbc4/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.708264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e0c1e8dbd04905b0d1963f4f00b275, user_id=18a6f40d6f9d48ebb71840e8abc0bbc4, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.715509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.776001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 901/1810] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:03 2022] PUT /identity/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a/users/e810aced3d47464caf483976573cf3e9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:03.777279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.777568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.777847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.802328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:03.806683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 910/1811] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:03 2022] PUT /identity/v3/projects/84e0c1e8dbd04905b0d1963f4f00b275/users/18a6f40d6f9d48ebb71840e8abc0bbc4/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:03.808194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:03.830329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:03.830329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:03.913599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5778-979a-491a-9fda-605482c2c0a8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:03.957459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:03.959064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.959633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.960274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:03.961710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:03.965842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efedac6-2811-4bb3-8dce-d477e6dca7d6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:03.987963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 911/1812] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:03.990346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175250be-9b05-46ea-b98a-b93efbea843f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:03.990901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175250be-9b05-46ea-b98a-b93efbea843f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:03.991349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175250be-9b05-46ea-b98a-b93efbea843f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.004768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 902/1813] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:04.007125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.054474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1c51-277b-4f0a-a150-910ee3850cfd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.062886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-175250be-9b05-46ea-b98a-b93efbea843f None None] MFA Rules not processed for user `18a6f40d6f9d48ebb71840e8abc0bbc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:04.064254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-175250be-9b05-46ea-b98a-b93efbea843f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:04.081982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.082888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.084260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.086525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] PATH_INFO: `/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a/users/e810aced3d47464caf483976573cf3e9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.090973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d542f66f40d4a1bb39a8f26f4563a0a, user_id=e810aced3d47464caf483976573cf3e9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.095244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.133685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 903/1814] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:04 2022] PUT /identity/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a/users/e810aced3d47464caf483976573cf3e9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:04.133685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.203393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 912/1815] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:04.206103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.206482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.209664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:04.211887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.268706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] MFA Rules not processed for user `e810aced3d47464caf483976573cf3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:04.270836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:04.283410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014900cd-b533-4a6f-bfd5-52507f803e4e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.289462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.290032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.290387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.290738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.291473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:04.291896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.293839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.319261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 904/1816] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:04.319261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.329863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3627d7-4647-4fab-b5a4-4603a2571fdc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.339719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.340660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.341002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.341220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.341952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.348478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.391725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 913/1817] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:04.393502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.411318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be523e55-4067-41f1-b936-54c7b105b36e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.419124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.420217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.420757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.421212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.422557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.430226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.560140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 905/1818] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:04 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:04.562839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.565850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:04.610630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 914/1819] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:04 2022] POST /identity/v3/projects => generated 369 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:04.612157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.624547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:04.674062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f135-2010-4da0-bbf3-2cc424f3ecad admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.758365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.759054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.759341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.759613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.763254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:04.768269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a36b24-3763-487a-bc0c-2639d8ab820a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:04.773602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:04.774713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:04.775196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:04.775801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:04.777828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:04.821053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:04.846497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.847974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.868424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 906/1820] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:04.869280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.874009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:04.876481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:04.893349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 915/1821] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:04.894433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:04.917007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:05.018321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd99b4-a8f9-421a-9cb2-d351edf040db service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.028408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.029164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.029447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.029721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.030358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.034560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.036019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d691-c758-492f-9aef-34e2736f11bf service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.042653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.043524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.044014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.045040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.045987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.049194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.152910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:05.193723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 907/1822] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:04 2022] POST /identity/v3/users => generated 288 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:05.202074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.204491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 916/1823] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:04 2022] POST /identity/v3/users => generated 285 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:05.209765 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.223624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663409f-0a2d-47d5-ae01-90319e1598a5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.259092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.259798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.260207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.261221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.261221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.262679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.272529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb70ad9-057b-4dec-b0a3-c3478dd220c4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.294847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.299441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.300238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.301205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.302711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.312521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.322901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 908/1824] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:05.340386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.363290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3097a1a-197f-43fa-b52c-acaa39edf3be admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.367638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 917/1825] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:05.370941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.377204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.378440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.379025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.401542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] PATH_INFO: `/v3/projects/151dc3686b844acca863ad54bf8b55f9/users/a3f6f5d59b984798ad051e51d4c52dc1/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.414529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf80d6c-9013-4c2e-952c-86a3fb02a971 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.437132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.438631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.441186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.441957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.448610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.460768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.462125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=151dc3686b844acca863ad54bf8b55f9, user_id=a3f6f5d59b984798ad051e51d4c52dc1, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.476966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.480448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:05.497483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 918/1826] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:05.501141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.577285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a077f-5c1b-4c48-9b84-37ce7095e66e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.590128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 909/1827] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:05 2022] PUT /identity/v3/projects/151dc3686b844acca863ad54bf8b55f9/users/a3f6f5d59b984798ad051e51d4c52dc1/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:05.596423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.785786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3bf43-952c-4d02-bf80-f50a2630b33f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.795703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.798908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.798908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.798908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.798908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.809631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.842567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.845109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.845600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 910/1828] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:05.845705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.846551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] PATH_INFO: `/v3/projects/de716ec8729a4fcc9aaa5e1fb2ff7caa/users/69896d97c0064c52a4cfd3533aa9fba5/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.851406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:05.864593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:05.874912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0393be1-6e1a-427f-ae69-8b87c7d7da4b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:05.904501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:05.906204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:05.906563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:05.909020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] PATH_INFO: `/v3/projects/151dc3686b844acca863ad54bf8b55f9/users/a3f6f5d59b984798ad051e51d4c52dc1/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:05.913914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de716ec8729a4fcc9aaa5e1fb2ff7caa, user_id=69896d97c0064c52a4cfd3533aa9fba5, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.916829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151dc3686b844acca863ad54bf8b55f9, user_id=a3f6f5d59b984798ad051e51d4c52dc1, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:05.921366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2520aaf-1cd8-42da-9f62-b5b485031e19 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.930532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fef37-c30f-42f9-ac48-b9994a50e1f5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:05.981983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 911/1829] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:05 2022] PUT /identity/v3/projects/151dc3686b844acca863ad54bf8b55f9/users/a3f6f5d59b984798ad051e51d4c52dc1/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:06.002238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:06.002757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:06.003131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:06.073658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 919/1830] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:05 2022] PUT /identity/v3/projects/de716ec8729a4fcc9aaa5e1fb2ff7caa/users/69896d97c0064c52a4cfd3533aa9fba5/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:06.106438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:06.108290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:06.109769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:06.111039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] MFA Rules not processed for user `a3f6f5d59b984798ad051e51d4c52dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:06.112470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:06.264322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] MFA Rules not processed for user `69896d97c0064c52a4cfd3533aa9fba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:06.299026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:06.352043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:06.379687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 912/1831] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:06.578048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 920/1832] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:06.591574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:06.650717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:06.855091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b98b65-bea5-4de1-93de-ce0717cd5192 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:06.871981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:06.873181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:06.873660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:06.874138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:06.906239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:06.911049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:06.926684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:06.997038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 921/1833] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:07.018973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5301-2269-44e6-8486-f66311c85fd0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.035987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.037529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.038057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.038445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.039238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.043688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.135331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:07.199776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 913/1834] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:06 2022] POST /identity/v3/projects => generated 369 bytes in 600 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:07.208157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.244299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42e0f2-d43c-48c7-aadd-7c29da798d68 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.259114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:07.325687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.326370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.326664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.326945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.327591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.329792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1a6c7-192f-4a5c-8372-16332f51cfd7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.404892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.480575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b827fb8-4326-4c2d-97a6-b47ca6da31a1 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.486363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.487123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.487512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.487931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.568617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.570139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.586718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 914/1835] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:07.601980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.607044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 922/1836] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:07 2022] POST /identity/v3/users => generated 291 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:07.609435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b690-af8d-4de4-afaf-b16bf3f3996f None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.610029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b690-af8d-4de4-afaf-b16bf3f3996f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.610612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9b690-af8d-4de4-afaf-b16bf3f3996f None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.613271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 923/1837] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:27:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:27:07.618669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3e13-c969-40a1-943b-fb1f5005b848 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.624113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.625142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.625625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.625995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.627541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.629291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.629974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.631409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.631993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.677231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 915/1838] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:07.683445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.683713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:07.687939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:07.726869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10590a-92ae-423e-ba91-eb689b3797dc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.738291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.740401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.740951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.741548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] PATH_INFO: `/v3/projects/2c3c80d3242842bcb7e63c5371415229/users/4352ab23e9a0491ea5c1a7b324b6e826/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.774526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3c80d3242842bcb7e63c5371415229, user_id=4352ab23e9a0491ea5c1a7b324b6e826, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.774526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.804286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 924/1839] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:07.817353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.822630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:07.832960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c5e61-4bcb-44f5-bb33-22c882ea2846 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:07.840266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:07.840875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:07.841163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:07.841441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:07.852971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:07.855411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:07.872638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 925/1840] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:07.931484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 916/1841] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:07 2022] PUT /identity/v3/projects/2c3c80d3242842bcb7e63c5371415229/users/4352ab23e9a0491ea5c1a7b324b6e826/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:07.938050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:07.970367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ac26c-ace1-4f65-b1f2-413ea42f8cf4 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:08.058440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:08.060389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.060389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.060389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.060389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:08.061948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f71ad-f06c-4382-93c8-f11e0b09b724 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:08.074150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 926/1842] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:08.078248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:08.089330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3628f9-e4e4-4996-bf99-6fab62498208 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:08.094483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:08.095204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.095485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.095907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] PATH_INFO: `/v3/projects/2c3c80d3242842bcb7e63c5371415229/users/4352ab23e9a0491ea5c1a7b324b6e826/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.098087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3c80d3242842bcb7e63c5371415229, user_id=4352ab23e9a0491ea5c1a7b324b6e826, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:08.101412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:08.117240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 917/1843] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:08 2022] PUT /identity/v3/projects/2c3c80d3242842bcb7e63c5371415229/users/4352ab23e9a0491ea5c1a7b324b6e826/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:08.140170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.140577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.141189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.188019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] MFA Rules not processed for user `4352ab23e9a0491ea5c1a7b324b6e826`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:08.192825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:08.290484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 927/1844] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:08.405462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:08.446102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:08.450526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:08.501954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a4c6-b400-4061-a1ba-ed5f317c1fc9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:08.506877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:08.507970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.508618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.524805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.524805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:08.524805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:08.534308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 918/1845] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:08.556705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f08384-5115-4bf2-b552-5b89defb63df None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:08.561705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:08.562364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.562651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.562923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] PATH_INFO: `/v3/users/d62f1ef589be4c25b36a5d7c65539643` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.564142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62f1ef589be4c25b36a5d7c65539643)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:08.566257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:08.713081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:08.730778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8640db7-3095-4681-85b9-7b2863c1e65d service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:08.734767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:08.735385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.735692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.735978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:08.749562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:08.811790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62f1ef589be4c25b36a5d7c65539643'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:08.823863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:08.825309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529155a-3981-4ec4-9fa7-ec38b3970e71 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:08.835863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 919/1846] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:08.910886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62f1ef589be4c25b36a5d7c65539643'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:08.912705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62f1ef589be4c25b36a5d7c65539643'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:08.913385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62f1ef589be4c25b36a5d7c65539643'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:08.929309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62f1ef589be4c25b36a5d7c65539643'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:08.967163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 928/1847] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:08 2022] DELETE /identity/v3/users/d62f1ef589be4c25b36a5d7c65539643 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:08.968782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:08.969325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:08.969976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:09.002994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:09.004833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:09.075733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 920/1848] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:09.086490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:09.088090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:09.159471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f922e-b796-4735-8bf2-f37579ce3113 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:09.167822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:09.168652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:09.169119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:09.169552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:09.186932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:09.186932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:09.198520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 929/1849] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:09.475098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:09.744671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:09.754649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:09.765743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d2d21-6a7f-42bd-9178-2e1b52aa8b8d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:09.788892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:09.789583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:09.790195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:09.790617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] PATH_INFO: `/v3/projects/3a65ff57d5f24ac1be16cdd442c181bd` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:09.793700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a65ff57d5f24ac1be16cdd442c181bd)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:09.795611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:09.903575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a65ff57d5f24ac1be16cdd442c181bd', 'request_id': 'req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:09.916811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a65ff57d5f24ac1be16cdd442c181bd', 'request_id': 'req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:09.991239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a65ff57d5f24ac1be16cdd442c181bd', 'request_id': 'req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.024995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] The token cache is being invalidate because project 3a65ff57d5f24ac1be16cdd442c181bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:10.025296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.026166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.027895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 921/1850] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:09 2022] DELETE /identity/v3/projects/3a65ff57d5f24ac1be16cdd442c181bd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:10.032537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:10.047270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78786a-c676-4ccc-bc16-d7b88e017ff8 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:10.174187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:10.175039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:10.175420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:10.175945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] PATH_INFO: `/v3/users/5b87abc01a0c4169b35025dca68206e2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:10.177334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b87abc01a0c4169b35025dca68206e2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:10.180079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:10.522637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b87abc01a0c4169b35025dca68206e2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.623299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b87abc01a0c4169b35025dca68206e2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.624007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b87abc01a0c4169b35025dca68206e2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.624622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b87abc01a0c4169b35025dca68206e2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.634599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e31238c3-feac-4b0a-b388-2dd0cb24f8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b87abc01a0c4169b35025dca68206e2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:10.664499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 930/1851] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:10 2022] DELETE /identity/v3/users/5b87abc01a0c4169b35025dca68206e2 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:10.701658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:10.701658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:10.724116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c34e7f-fca0-4df5-9bfd-021d1814ff7a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:10.891218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:10.892281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:10.892430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:10.892772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] PATH_INFO: `/v3/users/416cbb72fc6848edb1baafafec514d91` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:10.895318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=416cbb72fc6848edb1baafafec514d91)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:10.898995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:11.125227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416cbb72fc6848edb1baafafec514d91'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:11.171030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:11.234965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416cbb72fc6848edb1baafafec514d91'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:11.236210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416cbb72fc6848edb1baafafec514d91'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:11.237336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416cbb72fc6848edb1baafafec514d91'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:11.262738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416cbb72fc6848edb1baafafec514d91'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:11.347421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 922/1852] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:10 2022] DELETE /identity/v3/users/416cbb72fc6848edb1baafafec514d91 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:11.354763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:11.356308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:11.357503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:11.481006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:11.482263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:11.588920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:11.721773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 931/1853] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:11.756265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:11.763293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:11.768035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:11.777979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:11.876695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29931eab-7f4e-4e6c-9959-d0b81d845dae admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:11.931517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:11.937778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:11.938338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:11.938811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] PATH_INFO: `/v3/projects/95d03a896eb44a1da708252ba32158c8` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:11.945407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d03a896eb44a1da708252ba32158c8)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:11.953402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:12.072133 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221934b-aa08-48e6-9029-6ab4e9c00862 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:12.080480 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:12.081296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:12.081930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:12.082215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:12.082496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:12.109731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:12.111260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb49f36-b3bd-43de-af4d-e5b36435ad9a service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:12.149927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d03a896eb44a1da708252ba32158c8', 'request_id': 'req-9416b0da-9268-47f1-b02d-cb3a12701147'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:12.152144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 932/1854] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:12.179710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d03a896eb44a1da708252ba32158c8', 'request_id': 'req-9416b0da-9268-47f1-b02d-cb3a12701147'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:12.279120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d03a896eb44a1da708252ba32158c8', 'request_id': 'req-9416b0da-9268-47f1-b02d-cb3a12701147'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:12.332983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] The token cache is being invalidate because project 95d03a896eb44a1da708252ba32158c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:12.334027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:12.340218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9416b0da-9268-47f1-b02d-cb3a12701147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:12.344327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 923/1855] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:11 2022] DELETE /identity/v3/projects/95d03a896eb44a1da708252ba32158c8 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:12.353087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:12.353592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:12.353720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:12.403610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:12.407082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:12.414954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:12.756779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:12.757054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:12.777680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:12.798864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 933/1856] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:12.838953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:12.840697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:12.866404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 924/1857] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:12.872678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:12.888584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c9101-cdbe-47f0-8a3f-681cf319eb0a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:12.894826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:12.896263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:12.896562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:12.896841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:12.897590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:12.897898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:12.899534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:12.992318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 934/1858] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:12.997396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:13.029171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68006-3106-455d-a5f5-83aceef8da12 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:13.041791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:13.042914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:13.043674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:13.044172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:13.045188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:13.056550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:13.234175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 925/1859] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:12 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:13.235857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:13.259455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d165065-c1c6-4207-8602-dfda15a39d9c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:13.353538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:13.529632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:13.530558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:13.530906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:13.532306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:13.533741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:13.540330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:13.675241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 935/1860] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:13 2022] POST /identity/v3/users => generated 291 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:13.675999 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:13.705641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db7bcf-f555-42e1-8abb-319207015f3f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:13.728015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:13.728204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:13.728491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:13.728869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:13.729414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:13.731327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:13.775316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 926/1861] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:13.777735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:13.794326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96a6ef-6981-4e8c-b04f-e2e2278d0f48 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:13.800287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:13.801076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:13.801460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:13.802176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:13.803328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:13.805543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de0bdb-0760-472b-972a-2ae7a0a8d329 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:13.826690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:13.848259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 936/1862] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:13.857601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b7fc-bc6c-4417-81dd-cbea84ed5335 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:13.873359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:13.874367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:13.874763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:13.875141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] PATH_INFO: `/v3/projects/6a4836a0fdee4bc6a4e574e988877f07/users/b86a2f470266411bad767f90c644eb08/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:13.881666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:13.940498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4836a0fdee4bc6a4e574e988877f07, user_id=b86a2f470266411bad767f90c644eb08, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:13.943863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:14.047507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 927/1863] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:13 2022] PUT /identity/v3/projects/6a4836a0fdee4bc6a4e574e988877f07/users/b86a2f470266411bad767f90c644eb08/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:14.060446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:14.060787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:14.060994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:14.124766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] MFA Rules not processed for user `b86a2f470266411bad767f90c644eb08`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:14.128914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:14.237790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:14.237790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:14.397373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc3d0-c339-4dc2-9475-fbab54d95b83 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:14.397373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:14.397373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:14.397373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:14.398990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] PATH_INFO: `/v3/users/18a6f40d6f9d48ebb71840e8abc0bbc4` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:14.398990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a6f40d6f9d48ebb71840e8abc0bbc4)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:14.398990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:14.400755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 937/1864] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:14.400755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:14.408964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2ac6a-c53f-40e6-ac42-8d325abf652a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:14.415228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:14.416057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:14.416634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:14.417065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:14.417859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:14.419982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:14.681245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 938/1865] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:14 2022] POST /identity/v3/projects => generated 377 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:14.682631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:14.685313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:14.764469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a6f40d6f9d48ebb71840e8abc0bbc4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:14.785194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d289c-a779-49c2-b502-08a09e4c41b4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:14.792687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:14.793353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:14.793637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:14.793932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] PATH_INFO: `/v3/projects/4b1d52015b8a4bdaac64510b7d151ec7` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:14.796552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1d52015b8a4bdaac64510b7d151ec7)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:14.800847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:14.897660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a6f40d6f9d48ebb71840e8abc0bbc4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:14.898777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a6f40d6f9d48ebb71840e8abc0bbc4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:14.899813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a6f40d6f9d48ebb71840e8abc0bbc4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:14.929460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a6f40d6f9d48ebb71840e8abc0bbc4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:14.951682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 928/1866] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:14 2022] DELETE /identity/v3/users/18a6f40d6f9d48ebb71840e8abc0bbc4 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:14.964653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:14.964653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:14.980915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:14.994139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1d52015b8a4bdaac64510b7d151ec7', 'request_id': 'req-f0cca288-b681-48a1-93fc-671793712180'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.010987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1d52015b8a4bdaac64510b7d151ec7', 'request_id': 'req-f0cca288-b681-48a1-93fc-671793712180'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.094689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd20a8-b179-469d-a371-e70ba2623f96 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.122954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1d52015b8a4bdaac64510b7d151ec7', 'request_id': 'req-f0cca288-b681-48a1-93fc-671793712180'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.152962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] The token cache is being invalidate because project 4b1d52015b8a4bdaac64510b7d151ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:15.153885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.155347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f0cca288-b681-48a1-93fc-671793712180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.164113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 939/1867] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:14 2022] DELETE /identity/v3/projects/4b1d52015b8a4bdaac64510b7d151ec7 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:15.168648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.169453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.170976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.204687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.205406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.205754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.206198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.207093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.210025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.214244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:15.216943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:15.232847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 940/1868] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:15.234302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.238163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:15.257031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411ec12-7e75-4bf2-9618-bbbc4b66c21d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.262202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:15.271764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.272492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.272980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.273480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] PATH_INFO: `/v3/users/e810aced3d47464caf483976573cf3e9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.275735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e810aced3d47464caf483976573cf3e9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.278281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.305602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 929/1869] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:14 2022] POST /identity/v3/users => generated 295 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:15.306313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.308565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:15.428797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6f84d-94ea-4e7f-8391-2b542d67c305 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.435136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.435808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.436139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.436385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.447337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.449363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.462599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 930/1870] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:15.464910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.465594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:15.480320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34200de1-b021-46d1-9458-f353dc6a40d6 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.486378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.486378 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.486791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.486877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.487712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.489504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.502867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 931/1871] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:15.507827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.523936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589247c-5d8e-4c15-a0c7-48f2651ad3d9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.528913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.529583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.529872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.530260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] PATH_INFO: `/v3/projects/784ac146b1694af59d2ea18b052ceb08/users/57c6cd905d12461f97fac2e678561086/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.530404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e810aced3d47464caf483976573cf3e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.548811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784ac146b1694af59d2ea18b052ceb08, user_id=57c6cd905d12461f97fac2e678561086, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.551590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.625142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e810aced3d47464caf483976573cf3e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.628867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e810aced3d47464caf483976573cf3e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.628867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e810aced3d47464caf483976573cf3e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.644309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e810aced3d47464caf483976573cf3e9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:15.653995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 932/1872] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:15 2022] PUT /identity/v3/projects/784ac146b1694af59d2ea18b052ceb08/users/57c6cd905d12461f97fac2e678561086/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:15.655957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.659227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:15.676456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 941/1873] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:15 2022] DELETE /identity/v3/users/e810aced3d47464caf483976573cf3e9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:15.789289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448fbbe-99b4-4d03-8a1f-47ac1cce89d7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.853765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:15.917515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.918364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.918652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.918931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.919587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.921429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d999d9-0c27-45cc-9ca8-28e434780289 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:15.944122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 933/1874] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:15.948856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:15.964357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c13f9-36d9-4e73-8388-026a1de9932b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:15.974937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:15.976638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:15.977702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:15.978488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] PATH_INFO: `/v3/projects/784ac146b1694af59d2ea18b052ceb08/users/57c6cd905d12461f97fac2e678561086/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:15.981351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784ac146b1694af59d2ea18b052ceb08, user_id=57c6cd905d12461f97fac2e678561086, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:15.984808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:16.007786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 942/1875] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:15 2022] PUT /identity/v3/projects/784ac146b1694af59d2ea18b052ceb08/users/57c6cd905d12461f97fac2e678561086/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:16.019088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.019088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.019088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.053656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] MFA Rules not processed for user `57c6cd905d12461f97fac2e678561086`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:16.055290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.127942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:16.188312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:16.190305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.199849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 934/1876] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:16.227376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:16.229351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.323363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec69dd8-04dc-409e-a425-e51c2992494c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:16.324138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a2b1f-078c-4ec9-bc6a-aef4a5bfad89 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:16.328598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:16.329249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.329688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.329977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.331062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:16.332070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.332361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.332809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] PATH_INFO: `/v3/projects/84e0c1e8dbd04905b0d1963f4f00b275` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.334526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e0c1e8dbd04905b0d1963f4f00b275)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:16.336893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:16.341893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:16.343327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:16.354716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 935/1877] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:16.355733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:16.357633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.382417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fadc5-06ea-4aeb-956e-c34b5502ba7b service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:16.392621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:16.393803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.394261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.394731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] PATH_INFO: `/v3/users/69896d97c0064c52a4cfd3533aa9fba5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.397158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69896d97c0064c52a4cfd3533aa9fba5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:16.400689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:16.517595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e0c1e8dbd04905b0d1963f4f00b275', 'request_id': 'req-24070f96-7733-407b-be65-2d8b2d6a7718'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.528439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e0c1e8dbd04905b0d1963f4f00b275', 'request_id': 'req-24070f96-7733-407b-be65-2d8b2d6a7718'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.635065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e0c1e8dbd04905b0d1963f4f00b275', 'request_id': 'req-24070f96-7733-407b-be65-2d8b2d6a7718'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.649151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] The token cache is being invalidate because project 84e0c1e8dbd04905b0d1963f4f00b275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:16.649655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.650443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.652327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 943/1878] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:16 2022] DELETE /identity/v3/projects/84e0c1e8dbd04905b0d1963f4f00b275 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:16.653117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:16.654492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.658094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69896d97c0064c52a4cfd3533aa9fba5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.745207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69896d97c0064c52a4cfd3533aa9fba5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.745846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69896d97c0064c52a4cfd3533aa9fba5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.746459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69896d97c0064c52a4cfd3533aa9fba5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.756272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070f96-7733-407b-be65-2d8b2d6a7718 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:16.760034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69896d97c0064c52a4cfd3533aa9fba5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.761929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:16.762680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.762902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.763172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] PATH_INFO: `/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.765212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d542f66f40d4a1bb39a8f26f4563a0a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:16.767108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:16.777272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 936/1879] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:16 2022] DELETE /identity/v3/users/69896d97c0064c52a4cfd3533aa9fba5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:16.780969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:16.782350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:16.897071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b4e2-6cf6-4db4-96e5-cd03d013f8d1 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:16.903576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d542f66f40d4a1bb39a8f26f4563a0a', 'request_id': 'req-99b2302d-7401-4b10-9594-389ff60f64e8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.930678 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d542f66f40d4a1bb39a8f26f4563a0a', 'request_id': 'req-99b2302d-7401-4b10-9594-389ff60f64e8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.981521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d542f66f40d4a1bb39a8f26f4563a0a', 'request_id': 'req-99b2302d-7401-4b10-9594-389ff60f64e8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:16.995301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:16.996145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:16.996811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:16.997204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] PATH_INFO: `/v3/users/a3f6f5d59b984798ad051e51d4c52dc1` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:16.999244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f6f5d59b984798ad051e51d4c52dc1)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.002009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.003482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] The token cache is being invalidate because project 7d542f66f40d4a1bb39a8f26f4563a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:17.004214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.011784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:17.013846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-99b2302d-7401-4b10-9594-389ff60f64e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.018010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 944/1880] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:16 2022] DELETE /identity/v3/projects/7d542f66f40d4a1bb39a8f26f4563a0a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:17.026779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59fb9-d761-4291-9008-c400e931cf30 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.027281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59fb9-d761-4291-9008-c400e931cf30 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.030389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59fb9-d761-4291-9008-c400e931cf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.059253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-96e59fb9-d761-4291-9008-c400e931cf30 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:17.061028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-96e59fb9-d761-4291-9008-c400e931cf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.076124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 945/1881] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:17.080990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.080990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.080990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.113059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:17.114351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.137513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 946/1882] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:17.137513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.137513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.205246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f8dbd-fa27-4eca-92ca-52ddfe891ff1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.209661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.210280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.210563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.210838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.225542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.227102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773cc13-19b9-49de-9803-d917fdd12736 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.237726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 947/1883] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:17.239051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.239362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.239619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.262489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f6f5d59b984798ad051e51d4c52dc1'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.293432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:17.295430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.308431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 948/1884] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:17.314186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.324582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa63937-8de1-4bc7-9200-a55f6b065bcc None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.331782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.332796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.333355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.333963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.334872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:17.335278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.337190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.352980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 949/1885] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:27:17.357360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.368614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba2de-cf38-4553-8b78-edb7ea27d48d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.373812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.374469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.374719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.374991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.375637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.377569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.384001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f6f5d59b984798ad051e51d4c52dc1'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.386172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f6f5d59b984798ad051e51d4c52dc1'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.386495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f6f5d59b984798ad051e51d4c52dc1'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.408837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f6f5d59b984798ad051e51d4c52dc1'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:17.423321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 937/1886] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:16 2022] DELETE /identity/v3/users/a3f6f5d59b984798ad051e51d4c52dc1 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:17.478044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 950/1887] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:17.482394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.484369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.501578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:17.595730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.598273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:17.647607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188e7e-ae56-4424-b2eb-7e29b3a25be7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.714526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731f50e-c0a1-4361-a7aa-db103bbfb494 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.721957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c6cd905d12461f97fac2e678561086', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '784ac146b1694af59d2ea18b052ceb08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.722934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.723401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.724526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.725409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:17.729347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.729347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.743607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.744681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.745155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.745626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.746734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.752259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.757396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 951/1888] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:27:17.759602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.776952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786ad34-4a87-4355-a2c3-753efbc47b97 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.782260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c6cd905d12461f97fac2e678561086', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '784ac146b1694af59d2ea18b052ceb08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.782985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.783407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.783855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.788889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.788889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.877054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 938/1889] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/users => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:17.881219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.896603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2d52a-2448-4886-8291-1dd74b5dd9e1 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:17.905961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:17.905961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:17.906323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:17.906596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:17.907982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:17.909952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:17.934884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 939/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:17.936122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 952/1891] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:17 2022] POST /identity/v3/projects => generated 355 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:17.939268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.943519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:17.956790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf937-7448-4753-8063-cf03106d29ef admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.061575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097689f9-7298-4d00-8963-1bcf58976163 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.066182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.067162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c6cd905d12461f97fac2e678561086', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '784ac146b1694af59d2ea18b052ceb08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.067311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.068171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.068325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.068618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.068718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.068900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.069846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:18.070164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.070819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.071946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.073940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.093064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 953/1892] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:18.095942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.098368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:18.114798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 940/1893] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:18.116151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.129103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9684a5b-42e7-4d26-912e-d1a9757d506d tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.130526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7930a3-c26b-46f4-af0c-d31ed67921cc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.134985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.135932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.136393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.137454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] PATH_INFO: `/v3/projects/de716ec8729a4fcc9aaa5e1fb2ff7caa` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.138394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.139128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de716ec8729a4fcc9aaa5e1fb2ff7caa)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.139307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.139560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.139880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] PATH_INFO: `/v3/projects/70b6181cabf64bb09ad7ab57036efccb/users/f7e9b365655843cba4cb0a88a92352b1/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.141246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.171082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b6181cabf64bb09ad7ab57036efccb, user_id=f7e9b365655843cba4cb0a88a92352b1, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.174500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.262639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 941/1894] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:18 2022] PUT /identity/v3/projects/70b6181cabf64bb09ad7ab57036efccb/users/f7e9b365655843cba4cb0a88a92352b1/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:18.264377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.307184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de716ec8729a4fcc9aaa5e1fb2ff7caa', 'request_id': 'req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.341163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de716ec8729a4fcc9aaa5e1fb2ff7caa', 'request_id': 'req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.386254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445cc71-5cd6-4366-a255-127047cb9def admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.458703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c6cd905d12461f97fac2e678561086', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '784ac146b1694af59d2ea18b052ceb08', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.459233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.459527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.459828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] PATH_INFO: `/v3/projects/b438d1ce60fb44918d4b2cd2f3a8cb0f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.460976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b438d1ce60fb44918d4b2cd2f3a8cb0f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.462857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.465518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de716ec8729a4fcc9aaa5e1fb2ff7caa', 'request_id': 'req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.478740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] The token cache is being invalidate because project de716ec8729a4fcc9aaa5e1fb2ff7caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:18.479125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.479918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.482561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 954/1895] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:18 2022] DELETE /identity/v3/projects/de716ec8729a4fcc9aaa5e1fb2ff7caa => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:18.483775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.485346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:18.556625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:18.562110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f99fc2-2cbf-4d1e-a9d0-775bf8d790f1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.568491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.569589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.570429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.571582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] PATH_INFO: `/v3/projects/151dc3686b844acca863ad54bf8b55f9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.575613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151dc3686b844acca863ad54bf8b55f9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.577793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.606862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b438d1ce60fb44918d4b2cd2f3a8cb0f', 'request_id': 'req-96447ac1-21f5-4be8-b6dc-ec02a4a76405'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.612981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b438d1ce60fb44918d4b2cd2f3a8cb0f', 'request_id': 'req-96447ac1-21f5-4be8-b6dc-ec02a4a76405'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.688370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b438d1ce60fb44918d4b2cd2f3a8cb0f', 'request_id': 'req-96447ac1-21f5-4be8-b6dc-ec02a4a76405'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.702485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] The token cache is being invalidate because project b438d1ce60fb44918d4b2cd2f3a8cb0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:18.702801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.703695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96447ac1-21f5-4be8-b6dc-ec02a4a76405 tempest-VolumeQuotaClassesTest-79825384 tempest-VolumeQuotaClassesTest-79825384-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.709014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 942/1896] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:18 2022] DELETE /identity/v3/projects/b438d1ce60fb44918d4b2cd2f3a8cb0f => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:18.710509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.711007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.711505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.717691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151dc3686b844acca863ad54bf8b55f9', 'request_id': 'req-799702b2-dc57-4031-8de1-b424662d82d3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.729513 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151dc3686b844acca863ad54bf8b55f9', 'request_id': 'req-799702b2-dc57-4031-8de1-b424662d82d3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.750199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] MFA Rules not processed for user `f7e9b365655843cba4cb0a88a92352b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:18.751618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:18.827608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 943/1897] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:18.829095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.831392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:18.834417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151dc3686b844acca863ad54bf8b55f9', 'request_id': 'req-799702b2-dc57-4031-8de1-b424662d82d3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.848228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] The token cache is being invalidate because project 151dc3686b844acca863ad54bf8b55f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:18.848635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.849891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:18.851520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 955/1898] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:18 2022] DELETE /identity/v3/projects/151dc3686b844acca863ad54bf8b55f9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:18.852409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:18.853762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:18.936343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799702b2-dc57-4031-8de1-b424662d82d3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.941377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.942235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.943039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.943325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.944002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.944368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fd79e-6d01-4824-b65d-27f5c58b1bf9 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:18.945956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e01c9-8f3c-47b0-b307-e70425398ec9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:18.952127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:18.953147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:18.953620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:18.954083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] PATH_INFO: `/v3/users/4352ab23e9a0491ea5c1a7b324b6e826` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:18.956141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4352ab23e9a0491ea5c1a7b324b6e826)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:18.960122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:19.011595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:19.052252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 956/1899] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:18 2022] POST /identity/v3/projects => generated 401 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:19.052252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.052252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.052252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.086142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:19.089622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:19.200050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4352ab23e9a0491ea5c1a7b324b6e826'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:19.204511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 957/1900] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:19.204924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:19.226556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0838-dbab-45c9-8645-6c45237b742c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:19.231916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:19.239037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.239037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.239037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.239037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:19.239037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb4092-5ac3-43a4-9ed3-dbcd05890416 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:19.390280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:19.422182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 958/1901] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:19 2022] POST /identity/v3/users => generated 307 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:19.423631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.423955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.426440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.447047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4352ab23e9a0491ea5c1a7b324b6e826'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:19.448019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4352ab23e9a0491ea5c1a7b324b6e826'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:19.449221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4352ab23e9a0491ea5c1a7b324b6e826'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:19.456653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:19.458250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:19.483330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 959/1902] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:19.483330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:19.486102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:19.486522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4352ab23e9a0491ea5c1a7b324b6e826'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:19.514914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97b984-8340-4416-87aa-59a3dff7fd9a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:19.520005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 944/1903] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:18 2022] DELETE /identity/v3/users/4352ab23e9a0491ea5c1a7b324b6e826 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:19.523984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:19.654539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8196d5-7fa7-4787-8696-bbf267b969b0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:19.664870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:19.671134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:19.671977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.671977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.673502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.673502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.673502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.674627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.674627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:19.679957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:19.679957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:19.680173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:19.680719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:19.694207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 960/1904] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:19.695737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:19.700229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:19.723453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 945/1905] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:19.724667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:19.746712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aaf94-8087-441e-a97f-76f2b3876b97 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:19.754966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c288-856d-4453-acf6-c1a22d2ccf94 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:19.760306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:19.760991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.761273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.761546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] PATH_INFO: `/v3/users/b86a2f470266411bad767f90c644eb08` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.763239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86a2f470266411bad767f90c644eb08)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:19.765482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:19.773319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:19.774513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:19.775775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:19.776319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] PATH_INFO: `/v3/projects/98a15a2c35cb4c529a180d9b6786ce05/users/fa88eee63fc34ddd9d2906b3644fb9a9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:19.805425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a15a2c35cb4c529a180d9b6786ce05, user_id=fa88eee63fc34ddd9d2906b3644fb9a9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:19.816322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:20.804314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:20.818927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 946/1906] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:19 2022] PUT /identity/v3/projects/98a15a2c35cb4c529a180d9b6786ce05/users/fa88eee63fc34ddd9d2906b3644fb9a9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 1091 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:20.818927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:20.927953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:20.941784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86a2f470266411bad767f90c644eb08'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:20.967934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd795d0-7119-4b1a-8cae-598e6a4df29b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:20.973487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:20.974504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:20.974916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:20.975219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:20.976733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:20.977874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.053807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86a2f470266411bad767f90c644eb08'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:21.055159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86a2f470266411bad767f90c644eb08'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:21.056761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86a2f470266411bad767f90c644eb08'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:21.073632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86a2f470266411bad767f90c644eb08'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:21.104530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 947/1907] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:20 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:21.107537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:21.111643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:21.122969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 961/1908] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:19 2022] DELETE /identity/v3/users/b86a2f470266411bad767f90c644eb08 => generated 0 bytes in 1428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:21.124380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:21.129350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:21.276615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7732-17a3-4f01-8bb8-51b6618e0b4c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:21.356982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:21.387235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93b6a-8ce0-48f9-86b7-080d9d26dc82 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:21.391961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:21.392591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.392932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.394257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.396616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:21.398385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855faef9-55e3-4f03-bfaa-2f332fe02df9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.400136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:21.414003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.416723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.417331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.421697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:21.424213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.428140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 948/1909] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:21.433689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.437486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.438109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.471369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:21.472966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:21.493460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 949/1910] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:21.493460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:21.506649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276c92e-0dfe-43a5-8710-b87551b743c4 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:21.513199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:21.513889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.514192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.514628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] PATH_INFO: `/v3/projects/98a15a2c35cb4c529a180d9b6786ce05/users/fa88eee63fc34ddd9d2906b3644fb9a9/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.523701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 962/1911] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:21 2022] POST /identity/v3/users => generated 294 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:21.527959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:21.530235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a15a2c35cb4c529a180d9b6786ce05, user_id=fa88eee63fc34ddd9d2906b3644fb9a9, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:21.532913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.537608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:21.560821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 950/1912] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:21 2022] PUT /identity/v3/projects/98a15a2c35cb4c529a180d9b6786ce05/users/fa88eee63fc34ddd9d2906b3644fb9a9/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:21.574568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:21.696199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6863009-688d-4c48-a179-158b4329dcf7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:21.701979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32c881-bc9a-4b83-a032-4a29ccc0dc38 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:21.708594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:21.709341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.711852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.713675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.714421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:21.724130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:21.730374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98a3d1-0460-4147-bbb1-4c053df50bdb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.759471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 951/1913] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:21.762406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.768232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.768232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.849529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] MFA Rules not processed for user `fa88eee63fc34ddd9d2906b3644fb9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:21.852556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:21.881314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:21.882110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:21.882621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:21.885169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:21.930655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:21.932132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:21.998612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 963/1914] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:21.999680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.007801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.065712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de675a60-cc84-4765-b4da-de5c817c83d0 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:22.076862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 952/1915] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:22.076862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.096424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:22.097506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.098022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.098564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] PATH_INFO: `/v3/projects/2c3c80d3242842bcb7e63c5371415229` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.107547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b2fd4-d465-40ae-b302-3f93b9fe759e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:22.112454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3c80d3242842bcb7e63c5371415229)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:22.120637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:22.123009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.123009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.123353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.124189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:22.131519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:22.132052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:22.155509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 953/1916] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:22.155509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.156991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.195669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:22.283251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ed0c1-2167-4cb0-8eea-2544d7c7c7b4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:22.288330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:22.289080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.289289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.289570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.302316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:22.306303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:22.320626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 954/1917] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:22.321481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.335549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e20bab-dded-4d7f-897c-947f659dd772 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:22.348102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:22.349256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.349503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.349776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] PATH_INFO: `/v3/projects/77be5d9c80df4b6bb5a9dc654984535f/users/d9a2f10010924cff8823e631c5622b14/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.363490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3c80d3242842bcb7e63c5371415229', 'request_id': 'req-7f13629a-d883-405c-8b5e-933ae43da18a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:22.372262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3c80d3242842bcb7e63c5371415229', 'request_id': 'req-7f13629a-d883-405c-8b5e-933ae43da18a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:22.378051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77be5d9c80df4b6bb5a9dc654984535f, user_id=d9a2f10010924cff8823e631c5622b14, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:22.382103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958d413-93db-4521-bfc9-bd5f949bb495 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:22.516035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3c80d3242842bcb7e63c5371415229', 'request_id': 'req-7f13629a-d883-405c-8b5e-933ae43da18a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:22.530029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 955/1918] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:22 2022] PUT /identity/v3/projects/77be5d9c80df4b6bb5a9dc654984535f/users/d9a2f10010924cff8823e631c5622b14/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:22.548130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805a97-f1fb-473f-b39b-8f0fee3054fe None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.548521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805a97-f1fb-473f-b39b-8f0fee3054fe None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.548976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805a97-f1fb-473f-b39b-8f0fee3054fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.553439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] The token cache is being invalidate because project 2c3c80d3242842bcb7e63c5371415229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:22.554665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:22.558757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7f13629a-d883-405c-8b5e-933ae43da18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:22.564112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 964/1919] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:21 2022] DELETE /identity/v3/projects/2c3c80d3242842bcb7e63c5371415229 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:22.566714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.566978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.569182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.606581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:22.611305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.635500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8e805a97-f1fb-473f-b39b-8f0fee3054fe None None] MFA Rules not processed for user `d9a2f10010924cff8823e631c5622b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:22.637345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8e805a97-f1fb-473f-b39b-8f0fee3054fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.772455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 965/1920] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:22.773234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.781693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.906236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 956/1921] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:22.908262 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.908561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.908847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.934166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713354d3-1344-448b-b67d-b56f27e855d3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:22.939700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:22.942593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:22.943461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:22.944219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:22.946697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:22.948184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.949549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.961814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 957/1922] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:22.962878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:22.965658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:22.995884 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fbe39-8d14-4763-b3e9-92ab014b7e9a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.006147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.008213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.009742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.010244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.012873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.015497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.080338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.082259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.099125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 966/1923] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:23.100886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.103626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:23.119756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf322e9-88e9-41b3-bebb-3d59fa3dd30f service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.125672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.126411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.126794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.127149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] PATH_INFO: `/v3/projects/6a4836a0fdee4bc6a4e574e988877f07` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.129614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4836a0fdee4bc6a4e574e988877f07)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.132237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.192538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 958/1924] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:22 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:23.192538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.215251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:23.327539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0965d-c3c9-46f8-a113-bb8ee0c4e339 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.353309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.354409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.354898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.355186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.356182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:23.356585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.358221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.369122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4836a0fdee4bc6a4e574e988877f07', 'request_id': 'req-9b326d3c-42e7-4808-bc98-8de6c47333d8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.376950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 959/1925] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:23.376950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.381795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4836a0fdee4bc6a4e574e988877f07', 'request_id': 'req-9b326d3c-42e7-4808-bc98-8de6c47333d8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.392380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fc711-531a-4a57-8aaf-49f87b1a1086 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.464149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.464764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.465048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.465315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.465945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.467914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.491409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4836a0fdee4bc6a4e574e988877f07', 'request_id': 'req-9b326d3c-42e7-4808-bc98-8de6c47333d8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.503963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] The token cache is being invalidate because project 6a4836a0fdee4bc6a4e574e988877f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:23.504254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.505569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.507148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 967/1926] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:23 2022] DELETE /identity/v3/projects/6a4836a0fdee4bc6a4e574e988877f07 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:23.507952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.509252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:23.521989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b326d3c-42e7-4808-bc98-8de6c47333d8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.526947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.527547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.527860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.528212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.528797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.530781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.575211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 960/1927] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:23 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:23.576103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.577822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:23.590827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b712fd-6f1d-41df-8122-51e20945330d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.595684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.596319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.596607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.596887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] PATH_INFO: `/v3/users/57c6cd905d12461f97fac2e678561086` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.598033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c6cd905d12461f97fac2e678561086)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.600756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.650317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 968/1928] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:23 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:23.651193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.652778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:23.728216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d6bf5-502b-46b3-b92e-6a0c14095c89 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.733164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.733784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.734070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.734348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.734997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.736656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.750764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 969/1929] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:23.752064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.763684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a634c-37b4-4b3d-9c74-2c66a393aaf4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.768699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.769330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.769689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.769902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.771151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.773122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:23.816127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c6cd905d12461f97fac2e678561086'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.893626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 970/1930] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:23 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:23.894446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.908016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c6cd905d12461f97fac2e678561086'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.909116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483f17-5ab8-4223-b989-ba6e108acb55 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:23.910787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c6cd905d12461f97fac2e678561086'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.911672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c6cd905d12461f97fac2e678561086'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.930754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c6cd905d12461f97fac2e678561086'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:23.932480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:23.933197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:23.933490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:23.933875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] PATH_INFO: `/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446/users/313ff06f99cb4f139662f7dbea9059d3/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:23.964070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f644a2a06ef4d5eb36dc0250aa29446, user_id=313ff06f99cb4f139662f7dbea9059d3, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:23.965340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 961/1931] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:23 2022] DELETE /identity/v3/users/57c6cd905d12461f97fac2e678561086 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:23.966316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:23.969508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:23.969633 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.062255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 971/1932] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:23 2022] PUT /identity/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446/users/313ff06f99cb4f139662f7dbea9059d3/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:24.068809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.073613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:24.074172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756f4-d940-42ab-af37-c2a4bebbe151 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.174342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.175046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.176444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.176444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.176444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.180499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.205224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 962/1933] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:24.206488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.222708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b0559-05fc-40d3-804c-2a9e82aa632f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.228267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.228803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.229483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.230061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.230720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.232409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.237379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780b2a-6ba3-4d0d-8e8b-304a45bc87f5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.243973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.244610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.244899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.245180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.245910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.247680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.262934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 963/1934] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:24.264598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.268938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 972/1935] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:24.272577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.277363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5fc6-fcb7-4351-8005-ad1eb84d94c8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.283967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:24.285040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4518-babf-4c60-89c1-1f85cc0f28e8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.288594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.289382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.289788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.290159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] PATH_INFO: `/v3/projects/a436bf911f55435e98cac98c2feeb14e/users/38c8828944a0432ab63d5abf50ee5b09/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.290313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.291156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.291452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.291705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] PATH_INFO: `/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446/users/313ff06f99cb4f139662f7dbea9059d3/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.293699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f644a2a06ef4d5eb36dc0250aa29446, user_id=313ff06f99cb4f139662f7dbea9059d3, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.296446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0a470-39b1-4c61-8b05-479c854b7169 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.310094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 973/1936] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:24 2022] PUT /identity/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446/users/313ff06f99cb4f139662f7dbea9059d3/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:24.319503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a436bf911f55435e98cac98c2feeb14e, user_id=38c8828944a0432ab63d5abf50ee5b09, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.323917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb98e40-3f30-48b5-ad47-c99be9d6865a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.324653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.325026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.325377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.368980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] MFA Rules not processed for user `313ff06f99cb4f139662f7dbea9059d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:24.371015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:24.457878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 964/1937] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:24 2022] PUT /identity/v3/projects/a436bf911f55435e98cac98c2feeb14e/users/38c8828944a0432ab63d5abf50ee5b09/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:24.471214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.471707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.471996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.500788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 974/1938] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:24.507916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] MFA Rules not processed for user `38c8828944a0432ab63d5abf50ee5b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:24.510238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:24.518327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.519928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:24.644235 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a621977-a594-4ae4-85c1-64135b90def9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.644538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 965/1939] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:24.651272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.723387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.724543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.725223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.727059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:24.727893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.780707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e4391-705c-4209-a4f7-7c762203e786 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.787968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.789205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.789205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.789205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.791387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.792396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.817031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.818737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:24.830225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 975/1940] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:24.854784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.870957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:24.900300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bba0-1a74-4d75-b1d7-3ab4acce4423 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:24.920675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 966/1941] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:24 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:24.921440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:24.921607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:24.922048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:24.922377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] PATH_INFO: `/v3/projects/784ac146b1694af59d2ea18b052ceb08` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:24.925160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:24.934252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=784ac146b1694af59d2ea18b052ceb08)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:24.937826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.009633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b9ee5-7617-4b82-a597-0c1cfdb340df admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.015035 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.015804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.016096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.016398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.017015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.018926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.072353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784ac146b1694af59d2ea18b052ceb08', 'request_id': 'req-7cdd4e4e-8264-4f13-b957-b530624a5e6e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.079998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784ac146b1694af59d2ea18b052ceb08', 'request_id': 'req-7cdd4e4e-8264-4f13-b957-b530624a5e6e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.152269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 967/1942] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:24 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:25.153623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.166827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c2f8-b057-47b0-85a2-694a9225576a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.183188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784ac146b1694af59d2ea18b052ceb08', 'request_id': 'req-7cdd4e4e-8264-4f13-b957-b530624a5e6e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.197197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] The token cache is being invalidate because project 784ac146b1694af59d2ea18b052ceb08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:25.197486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.198177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.199767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 976/1943] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:24 2022] DELETE /identity/v3/projects/784ac146b1694af59d2ea18b052ceb08 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:25.204042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.205444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.243551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.244282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.244756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.245060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] PATH_INFO: `/v3/users/d9a2f10010924cff8823e631c5622b14` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.246633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a2f10010924cff8823e631c5622b14)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.248574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.305547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4e4e-8264-4f13-b957-b530624a5e6e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.311283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.312484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.312885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.313256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.313982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.316111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54192318-a899-4a9f-8a78-a7ac8e5540b4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.328565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 977/1944] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:25.330535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.330535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.330535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.355797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:25.357035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.367612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 978/1945] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:25.368793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.380885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7034d8-2cff-4f3e-82be-d6ff5462e916 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.388229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.389254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.389482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.389771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] PATH_INFO: `/v3/projects/92ee250a8708426ea614b8e9424f0fb5/users/e4feb2310a994a0eb064fd612e35cbf0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.391423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:25.408770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ee250a8708426ea614b8e9424f0fb5, user_id=e4feb2310a994a0eb064fd612e35cbf0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.411778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8a636-b85e-4e22-bb17-c15501a3b1f0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.470099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a2f10010924cff8823e631c5622b14'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.490333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 979/1946] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:25 2022] PUT /identity/v3/projects/92ee250a8708426ea614b8e9424f0fb5/users/e4feb2310a994a0eb064fd612e35cbf0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:25.491922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.492215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.492582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.519515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:25.521351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.569065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a2f10010924cff8823e631c5622b14'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.570258 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a2f10010924cff8823e631c5622b14'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.571241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a2f10010924cff8823e631c5622b14'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.594688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a2f10010924cff8823e631c5622b14'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:25.596617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 980/1947] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:25.608727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.611003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.624175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 968/1948] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:25 2022] DELETE /identity/v3/users/d9a2f10010924cff8823e631c5622b14 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:25.625236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.626997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363db6e-cd69-4d0c-86d8-e3c1c99766f7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.700133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d870-0ddb-4806-a540-82b453d94ee5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.705068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.706328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.706890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.707325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.708990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.711239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.711907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.712288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.712621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.713292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:25.713602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.715271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040af78b-f5bc-4571-92a5-77c02637a09e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.725220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.734217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 969/1949] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:25.736199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.736199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.741223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.751458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 981/1950] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:25.751458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.768052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:25.770055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686296de-d827-40d7-b2ab-db1695f50555 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.770242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.778681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.779310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.779580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.780067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.782068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.783898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 970/1951] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:25.785522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.790990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beddbb0-1aa7-496a-8f68-b3b31003931f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.819702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89eb66-47c1-4dfe-b462-3d32e4afc1c5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.827092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:25.827910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.828263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.829002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] PATH_INFO: `/v3/projects/92ee250a8708426ea614b8e9424f0fb5/users/e4feb2310a994a0eb064fd612e35cbf0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.830994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ee250a8708426ea614b8e9424f0fb5, user_id=e4feb2310a994a0eb064fd612e35cbf0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:25.839792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:25.871683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 971/1952] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:25 2022] PUT /identity/v3/projects/92ee250a8708426ea614b8e9424f0fb5/users/e4feb2310a994a0eb064fd612e35cbf0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:25.871683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:25.875079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:25.947000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 982/1953] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:25 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:25.947000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:25.947000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:25.947000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:25.970057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7d5a-8bc3-4a48-be46-c57751d2017d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:25.983265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] MFA Rules not processed for user `e4feb2310a994a0eb064fd612e35cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:25.984581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:26.033422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.034297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.034598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.034883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.061527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:26.081174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 983/1954] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:26.082440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.142034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.144882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.167978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 972/1955] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:26.170716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.172277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:26.189526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d256d-595c-462a-bb8d-02d85bfbb608 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:26.198357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.198631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.198916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.199490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.199917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.202569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.264719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4821c1-f1b3-48bc-a176-20d874b08e94 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:26.276533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.277909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.278457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.281755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.293229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.294790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.306510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 973/1956] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:26.340055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:26.389859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 984/1957] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:26 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:26.395832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.408394 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8c716-0189-490b-9280-7fe97aee1a23 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:26.414084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.414690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.414975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.415241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.415920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.418238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.432975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 974/1958] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:26.437106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.451072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4c414-230a-431e-a4cf-a61602a36a59 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:26.457139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.457884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.458771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.459149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.461005 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.461892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec819d-1601-45dd-9e93-4be2d47dbe8a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.498289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 985/1959] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:26.504770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.517087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e7fd4-1eef-4f90-be0d-b017ef4cedc7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:26.526551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:26.527445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.528808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.529585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] PATH_INFO: `/v3/projects/3fbb9140016540c2a6c18f13f8afd430/users/308e1a57c6d34e409dbe13c37e778d39/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.575403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbb9140016540c2a6c18f13f8afd430, user_id=308e1a57c6d34e409dbe13c37e778d39, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:26.583018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:26.681390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:26.749880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 975/1960] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:26 2022] PUT /identity/v3/projects/3fbb9140016540c2a6c18f13f8afd430/users/308e1a57c6d34e409dbe13c37e778d39/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:26.776107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea3516-b305-4649-ae58-7f108cf4ea16 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:26.776437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea3516-b305-4649-ae58-7f108cf4ea16 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:26.776933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea3516-b305-4649-ae58-7f108cf4ea16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:26.847065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2cea3516-b305-4649-ae58-7f108cf4ea16 None None] MFA Rules not processed for user `308e1a57c6d34e409dbe13c37e778d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:26.847625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2cea3516-b305-4649-ae58-7f108cf4ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:26.860498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:26.880107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:27.035079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a20e1-19ae-4370-9317-f38e76ee2cdb admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:27.049488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:27.050157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.050442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.050722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] PATH_INFO: `/v3/projects/77be5d9c80df4b6bb5a9dc654984535f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.053328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77be5d9c80df4b6bb5a9dc654984535f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:27.059057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:27.066043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:27.188224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 986/1961] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:27.190628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa525c-9535-43cd-9e8c-e688eef13f88 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.191116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa525c-9535-43cd-9e8c-e688eef13f88 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.195939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa525c-9535-43cd-9e8c-e688eef13f88 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.199122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 987/1962] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:27:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:27:27.200937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa525c-9535-43cd-9e8c-e688eef13f88 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:27.274799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77be5d9c80df4b6bb5a9dc654984535f', 'request_id': 'req-d4a7ad88-77a8-4330-84cf-12cecc035eb5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:27.278898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa525c-9535-43cd-9e8c-e688eef13f88 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:27.294285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77be5d9c80df4b6bb5a9dc654984535f', 'request_id': 'req-d4a7ad88-77a8-4330-84cf-12cecc035eb5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:27.378153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:27.379128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.379609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.380787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.381897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:27.385483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:27.462144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77be5d9c80df4b6bb5a9dc654984535f', 'request_id': 'req-d4a7ad88-77a8-4330-84cf-12cecc035eb5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:27.474663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] The token cache is being invalidate because project 77be5d9c80df4b6bb5a9dc654984535f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:27.474971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:27.475678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4a7ad88-77a8-4330-84cf-12cecc035eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:27.477554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 976/1963] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:26 2022] DELETE /identity/v3/projects/77be5d9c80df4b6bb5a9dc654984535f => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:27.478649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.478936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.479215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.500311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:27.502065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:27.603158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 988/1964] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:27 2022] POST /identity/v3/projects => generated 367 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:27.604776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:27.609032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:27.654678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 977/1965] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:27:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:27.656064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:27.657819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:27.674208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b660212-3200-4fb4-9ed2-301fe80c9952 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:27.712858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:27.715941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647161-f499-4f55-bac4-76a6be1c0003 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:27.722244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:27.723314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.723314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.723526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] PATH_INFO: `/v3/users/38c8828944a0432ab63d5abf50ee5b09` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.724697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c8828944a0432ab63d5abf50ee5b09)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:27.733596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:27.760938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:27.763404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:27.763404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:27.763404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] PATH_INFO: `/v3/users/313ff06f99cb4f139662f7dbea9059d3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:27.763880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313ff06f99cb4f139662f7dbea9059d3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:27.765892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:27.804674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:28.127050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c8828944a0432ab63d5abf50ee5b09'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.348873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c8828944a0432ab63d5abf50ee5b09'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.348873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c8828944a0432ab63d5abf50ee5b09'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.348873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c8828944a0432ab63d5abf50ee5b09'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.382211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c8828944a0432ab63d5abf50ee5b09'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.411576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313ff06f99cb4f139662f7dbea9059d3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.462072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 989/1966] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:27 2022] DELETE /identity/v3/users/38c8828944a0432ab63d5abf50ee5b09 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:28.463514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:28.465741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:28.608221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313ff06f99cb4f139662f7dbea9059d3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.612106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313ff06f99cb4f139662f7dbea9059d3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.612866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e688a3-1e33-4094-8c57-9b5abfb7944f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:28.612991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313ff06f99cb4f139662f7dbea9059d3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.625315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:28.625315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:28.627772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:28.627898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:28.628516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:28.633144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ddb9d-2f25-4a60-bff9-371d7ec8a680 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:28.649138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313ff06f99cb4f139662f7dbea9059d3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:28.685875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 978/1967] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:27 2022] DELETE /identity/v3/users/313ff06f99cb4f139662f7dbea9059d3 => generated 0 bytes in 1028 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:28.687546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:28.688965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:28.754294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 990/1968] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:28 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:28.757719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:28.758436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:28.758867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:28.816922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:28.820216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:28.892925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6eb3e-4e38-4e9d-9abd-9c84b3df6e85 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:28.915953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:28.922916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:28.922916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:28.922916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:28.939876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:28.979107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 991/1969] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:28.980595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:28.983974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:28.985508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:28.987044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:28.987596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.000942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7a16-3366-4ef3-8d68-881ded8f7b0f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.018245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.019148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.020887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.022096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.022443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 979/1970] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:29.023097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.023550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.025585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.028727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:29.041801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 992/1971] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:29.049225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.064198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cfbc-bca2-416b-b450-99df7c1cd466 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.113676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.122679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.126562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.131405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] PATH_INFO: `/v3/projects/3441435986c342a3b98e570d273ae318/users/55821b67b26f4dc891ed870a111457ad/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.179988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3441435986c342a3b98e570d273ae318, user_id=55821b67b26f4dc891ed870a111457ad, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.182126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee167e2-8eeb-4ed1-8aaf-290d34f913b8 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.184245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.194983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.195621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.197192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.197555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.220781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.222288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.248096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 980/1972] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:29.362592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 993/1973] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:29 2022] PUT /identity/v3/projects/3441435986c342a3b98e570d273ae318/users/55821b67b26f4dc891ed870a111457ad/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:29.367523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.378390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b3bd-1190-4c4f-bcaa-535f96042c20 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.509424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.511314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.511542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.512311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.514069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.517579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467be1c-004d-474a-833f-993a2bc6de88 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.542434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 981/1974] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:29.549090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.570849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19593235-4ee3-495c-bf00-22a085df2832 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.577259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.578997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.579275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.579842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] PATH_INFO: `/v3/projects/3441435986c342a3b98e570d273ae318/users/55821b67b26f4dc891ed870a111457ad/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.582482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3441435986c342a3b98e570d273ae318, user_id=55821b67b26f4dc891ed870a111457ad, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.585764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.597232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 994/1975] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:29 2022] PUT /identity/v3/projects/3441435986c342a3b98e570d273ae318/users/55821b67b26f4dc891ed870a111457ad/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:29.609321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.609616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.609922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.646218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] MFA Rules not processed for user `55821b67b26f4dc891ed870a111457ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:29.651299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:29.689667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.691863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:29.798668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598729a-7e83-4bad-adad-0fc80c338802 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.805613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.806299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.806608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.806904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] PATH_INFO: `/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.808147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f644a2a06ef4d5eb36dc0250aa29446)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.810156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.812871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 982/1976] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:29.825432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:29.833989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:29.837391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:29.944356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506a0a3-a335-450a-838a-d7dfddcbdb7f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:29.949372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:29.950039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:29.950328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:29.952909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:29.962026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:29.963473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:29.976208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 983/1977] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:30.067831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f644a2a06ef4d5eb36dc0250aa29446', 'request_id': 'req-53a6896e-661b-49c2-b07d-362a3e43d641'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.078200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f644a2a06ef4d5eb36dc0250aa29446', 'request_id': 'req-53a6896e-661b-49c2-b07d-362a3e43d641'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.179730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:30.182749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.203525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b295-e115-45e1-8c5e-b5b099068530 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:30.279616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f644a2a06ef4d5eb36dc0250aa29446', 'request_id': 'req-53a6896e-661b-49c2-b07d-362a3e43d641'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.292791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:30.293441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.293885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.294166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] PATH_INFO: `/v3/projects/a436bf911f55435e98cac98c2feeb14e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.295341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a436bf911f55435e98cac98c2feeb14e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:30.295733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] The token cache is being invalidate because project 4f644a2a06ef4d5eb36dc0250aa29446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:30.296035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.296729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53a6896e-661b-49c2-b07d-362a3e43d641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.297684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:30.302480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 995/1978] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:29 2022] DELETE /identity/v3/projects/4f644a2a06ef4d5eb36dc0250aa29446 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:30.304437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.304911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.305280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.355974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:30.357533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.368529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 996/1979] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:30.369541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:30.370964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.432737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da40933-5c28-4745-a9f8-a53f37b83414 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:30.436444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:30.437069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.437352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.437627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.439412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.511679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:30.513326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8037ce5-5845-4e6f-9f26-ca25ad591f92 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:30.530673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 997/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:30.531940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.532239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.532541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.562227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:30.562452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a436bf911f55435e98cac98c2feeb14e', 'request_id': 'req-668369cc-2c3e-4435-bedc-aa276c2c2ad8'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.563732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.570131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a436bf911f55435e98cac98c2feeb14e', 'request_id': 'req-668369cc-2c3e-4435-bedc-aa276c2c2ad8'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.647508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 998/1981] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:30.648364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:30.650263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.664303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599dda-988e-4443-8b32-d623d572b090 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:30.669623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:30.671130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.671130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.671130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] PATH_INFO: `/v3/users/f7e9b365655843cba4cb0a88a92352b1` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.672431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e9b365655843cba4cb0a88a92352b1)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:30.675643 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:30.676557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a436bf911f55435e98cac98c2feeb14e', 'request_id': 'req-668369cc-2c3e-4435-bedc-aa276c2c2ad8'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.691419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] The token cache is being invalidate because project a436bf911f55435e98cac98c2feeb14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:30.691860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.692906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:30.697966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 984/1982] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:30 2022] DELETE /identity/v3/projects/a436bf911f55435e98cac98c2feeb14e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:30.699004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:30.700611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.721877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668369cc-2c3e-4435-bedc-aa276c2c2ad8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:30.724228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:30.724940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.725332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.725332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.726321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:30.726692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:30.728450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:30.750632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 985/1983] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:30.750632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:30.750632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:30.769095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f80075-cd09-4e6e-bc08-cb33dc98f1dc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:30.774390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:30.775161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:30.775418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:30.775729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] PATH_INFO: `/v3/users/e4feb2310a994a0eb064fd612e35cbf0` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:30.776934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4feb2310a994a0eb064fd612e35cbf0)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:30.778862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.035381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e9b365655843cba4cb0a88a92352b1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.089206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4feb2310a994a0eb064fd612e35cbf0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.233876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e9b365655843cba4cb0a88a92352b1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.234542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e9b365655843cba4cb0a88a92352b1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.235163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e9b365655843cba4cb0a88a92352b1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.243616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e9b365655843cba4cb0a88a92352b1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.263687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 999/1984] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:30 2022] DELETE /identity/v3/users/f7e9b365655843cba4cb0a88a92352b1 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:31.267523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.268959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:31.339905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe28f4-c1f6-4313-845f-e8d84dc2cb9e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.344756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.345387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.345672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.345948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.346588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.348566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.384923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4feb2310a994a0eb064fd612e35cbf0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.389074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4feb2310a994a0eb064fd612e35cbf0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.389074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4feb2310a994a0eb064fd612e35cbf0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.402298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f1d9022f-447e-49bc-8bdd-abadff4fb772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4feb2310a994a0eb064fd612e35cbf0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:31.416614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 986/1985] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:30 2022] DELETE /identity/v3/users/e4feb2310a994a0eb064fd612e35cbf0 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:31.418225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.418565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.418968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.440660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:31.441280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:31.510750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 987/1986] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:31.521132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.522690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:31.549777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1000/1987] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:31 2022] POST /identity/v3/projects => generated 385 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:31.550947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.552807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:31.596667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0f261-b52f-4b4c-95a8-0585ce7e81a0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.628260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9872-e292-4c3c-bab0-7ffc3607f5a8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.634382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.635272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.639467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.639856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.640533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.642529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.650524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.651303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.651673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.652112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.662452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.664031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.674433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 988/1988] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:31.862906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1001/1989] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:31 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:31.867304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.876981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a743c-a41c-43f6-aa8b-ef0f0b825601 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.881590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.882201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.882485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.882762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.883453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.885189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.896289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 989/1990] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:31.901394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.914141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c09e7-dba8-47e2-9747-d8b41f88274b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.919278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.919947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.920231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.920509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.921158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.923236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:31.937320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:31.951040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1002/1991] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:31.952293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:31.966596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2ab-527f-47e6-ab2f-c5a41c38ede0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:31.974148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:31.975776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:31.975891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:31.976223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] PATH_INFO: `/v3/projects/062fe0bdb0894b4cab318f203d39534d/users/4e1bc3f6ea7f4c898bd43f22e3b2bac1/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:31.994149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=062fe0bdb0894b4cab318f203d39534d, user_id=4e1bc3f6ea7f4c898bd43f22e3b2bac1, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:31.996944 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e9985-bdc5-4c49-bbee-815c20158c6f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:32.138394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:32.140242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:32.154742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13148d12-65f1-477e-a8d5-b1afa1b821e9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:32.160275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:32.160929 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.161213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.161490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] PATH_INFO: `/v3/projects/92ee250a8708426ea614b8e9424f0fb5` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.162647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ee250a8708426ea614b8e9424f0fb5)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:32.164606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:32.274757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 990/1992] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:31 2022] PUT /identity/v3/projects/062fe0bdb0894b4cab318f203d39534d/users/4e1bc3f6ea7f4c898bd43f22e3b2bac1/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:32.285045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.285351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.285661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.314685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] MFA Rules not processed for user `4e1bc3f6ea7f4c898bd43f22e3b2bac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:32.315940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:32.377385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 991/1993] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:32.381828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:32.402043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ee250a8708426ea614b8e9424f0fb5', 'request_id': 'req-609c849e-0cc4-4fd9-a16c-8bad55d62b01'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.408401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ee250a8708426ea614b8e9424f0fb5', 'request_id': 'req-609c849e-0cc4-4fd9-a16c-8bad55d62b01'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.448600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b69c57-2f9d-4777-b9c5-5a5425cda75a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:32.530999 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:32.531806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.532195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.532566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.533469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:32.536331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea547e-769a-4d17-8b1c-92d85fea320c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:32.619399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ee250a8708426ea614b8e9424f0fb5', 'request_id': 'req-609c849e-0cc4-4fd9-a16c-8bad55d62b01'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.630352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] The token cache is being invalidate because project 92ee250a8708426ea614b8e9424f0fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:32.630641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.631268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.632786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1003/1994] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:32 2022] DELETE /identity/v3/projects/92ee250a8708426ea614b8e9424f0fb5 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:32.634302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:32.635587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:32.648834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c849e-0cc4-4fd9-a16c-8bad55d62b01 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:32.653357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:32.653971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.654251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.654525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] PATH_INFO: `/v3/projects/70b6181cabf64bb09ad7ab57036efccb` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.655628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b6181cabf64bb09ad7ab57036efccb)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:32.657549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:32.735492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 992/1995] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:32 2022] POST /identity/v3/projects => generated 387 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:32.736391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.736744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.737023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.758594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:32.759858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:32.828045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b6181cabf64bb09ad7ab57036efccb', 'request_id': 'req-099e0a82-0000-4043-b841-9badfe2a19ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.849167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b6181cabf64bb09ad7ab57036efccb', 'request_id': 'req-099e0a82-0000-4043-b841-9badfe2a19ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:32.852723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 993/1996] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:32.853720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:32.855281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:32.869328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ec61c-0c56-48ed-8870-5c02dc21730c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:32.934769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:32.935474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:32.935797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:32.936223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:32.936881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:32.938747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.005167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b6181cabf64bb09ad7ab57036efccb', 'request_id': 'req-099e0a82-0000-4043-b841-9badfe2a19ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.017027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] The token cache is being invalidate because project 70b6181cabf64bb09ad7ab57036efccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:33.017318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.017957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-099e0a82-0000-4043-b841-9badfe2a19ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.019539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1004/1997] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:32 2022] DELETE /identity/v3/projects/70b6181cabf64bb09ad7ab57036efccb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:33.020858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.021149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.021428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.041648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:33.042811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:33.052024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1005/1998] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:33.052881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.054605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:33.063364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:33.068966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e8aad-dea4-4ab3-b887-fd87f0f8ade4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.074634 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.075740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.075740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.075914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] PATH_INFO: `/v3/users/fa88eee63fc34ddd9d2906b3644fb9a9` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.077027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa88eee63fc34ddd9d2906b3644fb9a9)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.079082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.155437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 994/1999] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:32 2022] POST /identity/v3/users => generated 294 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:33.155437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.164469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2714b2-3ca9-4e56-bb1b-ce0720c3bfdf admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.169767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.170396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.170717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.171019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.171751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:33.172087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.173713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.191247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 995/2000] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:33.192227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.194518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:33.208804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a4677-9415-49c4-bd20-fee2212484cf admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.214502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.215207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.215613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.215973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.216690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.218458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.229691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 996/2001] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:33.229882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.241637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a729c-504e-4897-9ee2-0580dfb7f083 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.246416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.247028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.247337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.247620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.248291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.250237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.579853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 997/2002] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:33 2022] POST /identity/v3/projects => generated 385 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:33.580647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.594873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa88eee63fc34ddd9d2906b3644fb9a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.652309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41616d-de26-4257-a2e2-1eb7bf9d5406 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.657910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.658587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.658910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.659187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.680206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.680206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.680206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 998/2003] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:33.680206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.684863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f3e8b-77ca-472c-a58d-3b601fecc464 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.692024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.692845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.693135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.693954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.694613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.696581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.794740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa88eee63fc34ddd9d2906b3644fb9a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.795682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa88eee63fc34ddd9d2906b3644fb9a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.796367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa88eee63fc34ddd9d2906b3644fb9a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.804915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa88eee63fc34ddd9d2906b3644fb9a9'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:33.832799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1006/2004] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:33 2022] DELETE /identity/v3/users/fa88eee63fc34ddd9d2906b3644fb9a9 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:33.832799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.832799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:33.930163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b6603-e17a-4c3f-aa5d-2bac8187d092 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.936098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:33.938231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:33.938231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:33.938231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] PATH_INFO: `/v3/projects/43295fd82b56469c8a4957dca2855478/users/df89414a7e95448197dd77b360cf18bb/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:33.957107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43295fd82b56469c8a4957dca2855478, user_id=df89414a7e95448197dd77b360cf18bb, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:33.960316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a30e0-6311-4c34-a58f-3580057fc1a8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:33.968269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 999/2005] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:33 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:33.972836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:33.975996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:33.991296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664550a-4585-4023-ae38-1155c585e70b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:33.999986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.000874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.001339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.001624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.002280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.012669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.027691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1000/2006] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:34.030322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.050084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf366184-1a54-44eb-aa73-83490dbb3a23 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:34.057316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.058055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.062108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.062108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.062108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.062108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.079280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1001/2007] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:34.084868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.105127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0d07-8d11-4707-9f4d-f7d778465b77 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:34.107932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1007/2008] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:33 2022] PUT /identity/v3/projects/43295fd82b56469c8a4957dca2855478/users/df89414a7e95448197dd77b360cf18bb/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:34.111415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.111696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.111987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.147549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] MFA Rules not processed for user `df89414a7e95448197dd77b360cf18bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:34.149251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:34.211793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.212692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.212984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.213267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] PATH_INFO: `/v3/projects/9e11882139da477398c7c95a5933669a/users/347153b838034dcebf6c1f804ae47925/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.267477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1008/2009] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:34.270348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.281132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e11882139da477398c7c95a5933669a, user_id=347153b838034dcebf6c1f804ae47925, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.283117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39adb8-590d-4dd6-b8a1-77540fb411f3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:34.284491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca3b3-43c0-476e-955a-d577744d13f4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.292934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.293560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.293896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.294173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.294870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.296853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.479267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1002/2010] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:34 2022] PUT /identity/v3/projects/9e11882139da477398c7c95a5933669a/users/347153b838034dcebf6c1f804ae47925/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:34.490178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.491156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.491502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.522365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] MFA Rules not processed for user `347153b838034dcebf6c1f804ae47925`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:34.523687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:34.589254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1003/2011] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:34.593931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.621092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1009/2012] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:34 2022] POST /identity/v3/projects => generated 387 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:34.626081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.658166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04641e-8406-4896-8251-25ad941a2268 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:34.692444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105776a3-a04c-400f-9d38-82ed60685762 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:34.702308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.703289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.703841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.704294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.705428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.708182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.728701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:34.730319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:34.730319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:34.730319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:34.732597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:34.732597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:34.885151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1010/2013] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:34 2022] POST /identity/v3/users => generated 300 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:34.886239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.889348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:34.899686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1004/2014] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:34 2022] POST /identity/v3/projects => generated 383 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:34.904279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:34.904460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaec607-d200-4c4c-bd27-7971abc030e7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.005418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd73b-9981-4f29-bfba-8956a891648d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.010501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.011147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.011433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.011735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.012382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.013971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.014089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.014682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.015000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.015318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] PATH_INFO: `/v3/projects/98a15a2c35cb4c529a180d9b6786ce05` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.016490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a15a2c35cb4c529a180d9b6786ce05)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.018521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.025022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1005/2015] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:35.025834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.035418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c63615-8386-4bcb-b346-ba7ed2d98a26 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.041023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.041620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.041902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.042174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.042799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.044741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.198700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1006/2016] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:35 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:35.199470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.210124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7610efd-4931-4d0a-94ce-1359a1367d7b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.212856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a15a2c35cb4c529a180d9b6786ce05', 'request_id': 'req-206e21a6-ed9d-419f-958a-78316fe06f84'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:35.220944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a15a2c35cb4c529a180d9b6786ce05', 'request_id': 'req-206e21a6-ed9d-419f-958a-78316fe06f84'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:35.269127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.269799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.270083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.270417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] PATH_INFO: `/v3/projects/cffe2374b2e94655a9d76227805c3b31/users/b22eae5a0b1d4bb4ab1d5a89837e454a/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.285813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffe2374b2e94655a9d76227805c3b31, user_id=b22eae5a0b1d4bb4ab1d5a89837e454a, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.288864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd750d-ed72-466d-9362-36621787d977 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.351030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a15a2c35cb4c529a180d9b6786ce05', 'request_id': 'req-206e21a6-ed9d-419f-958a-78316fe06f84'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:35.367128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] The token cache is being invalidate because project 98a15a2c35cb4c529a180d9b6786ce05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:35.367428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:35.368162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:35.370378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1011/2017] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:34 2022] DELETE /identity/v3/projects/98a15a2c35cb4c529a180d9b6786ce05 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:35.370966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.372827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.388529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e21a6-ed9d-419f-958a-78316fe06f84 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.394582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.395266 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.395969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.397105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.397761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.399379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4373b7-9cd4-4112-a8ee-d04c2f5cbde8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.410669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1012/2018] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:35.411954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.413713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.413713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.434032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:35.435241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.445795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1013/2019] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:35.445795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.459275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6e4f-0ae8-405a-b3ba-900de7a1bf8d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.460375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1007/2020] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:35 2022] PUT /identity/v3/projects/cffe2374b2e94655a9d76227805c3b31/users/b22eae5a0b1d4bb4ab1d5a89837e454a/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:35.461909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.462032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.462563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.493311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:35.495382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.561777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.562437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.562768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.563104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] PATH_INFO: `/v3/projects/830b9996c1714040840268f88436fa00/users/a2ab10cde2d74c95b11a1795cc327790/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.581896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830b9996c1714040840268f88436fa00, user_id=a2ab10cde2d74c95b11a1795cc327790, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.585728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018201f6-dee2-4b23-b8fd-56037eb55d67 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.602712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1008/2021] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:35.602712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.602712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.614006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79381a81-4c43-47c0-bac4-0019b53f98fd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.619959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.620661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.621015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.621451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.622113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.623811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.634806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1009/2022] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:35.635829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.645834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b10f-303e-43b9-ac76-17548958707b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.650765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.651441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.651759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.652043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.652698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:35.653006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.654595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.668753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1010/2023] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:35.669594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.681090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6eb1-c40a-40bc-9fc8-c09bc432d10d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.685867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.686503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.686786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.687060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] PATH_INFO: `/v3/projects/cffe2374b2e94655a9d76227805c3b31/users/b22eae5a0b1d4bb4ab1d5a89837e454a/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.689044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffe2374b2e94655a9d76227805c3b31, user_id=b22eae5a0b1d4bb4ab1d5a89837e454a, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.691640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.702559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1011/2024] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:35 2022] PUT /identity/v3/projects/cffe2374b2e94655a9d76227805c3b31/users/b22eae5a0b1d4bb4ab1d5a89837e454a/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:35.703415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.742640 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1014/2025] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:35 2022] PUT /identity/v3/projects/830b9996c1714040840268f88436fa00/users/a2ab10cde2d74c95b11a1795cc327790/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:35.746103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.746103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.746103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.773227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492921-a718-4c1b-b7e4-4f1c67316064 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.775283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] MFA Rules not processed for user `b22eae5a0b1d4bb4ab1d5a89837e454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:35.777190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.837377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.838052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.838339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.838618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.839281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.841579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.857506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1015/2026] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:35.858366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.870230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775da3e-77a7-48bd-aaef-11093d562b46 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.874799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.875396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.875704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.875979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.876618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.878199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.888626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1016/2027] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:35.889407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.890983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:35.949390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d207-537b-43bd-baa1-7c431eb93a86 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:35.953097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:35.953700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:35.953983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:35.954258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:35.963907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:35.966963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:35.975037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1017/2028] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:35.975873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:35.986567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78447a-ccc8-44c5-9ade-f23d18839454 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.005134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:36.014579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.014802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1012/2029] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:35 2022] POST /identity/v3/projects => generated 409 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:36.020267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.020267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.020267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] PATH_INFO: `/v3/projects/830b9996c1714040840268f88436fa00/users/a2ab10cde2d74c95b11a1795cc327790/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.020645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:36.020725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830b9996c1714040840268f88436fa00, user_id=a2ab10cde2d74c95b11a1795cc327790, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.024603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d0ec2-b9cd-4f9e-a8ee-cd3876662a54 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.051276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1018/2030] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:35 2022] PUT /identity/v3/projects/830b9996c1714040840268f88436fa00/users/a2ab10cde2d74c95b11a1795cc327790/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:36.068216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.069083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.069366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.106068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] MFA Rules not processed for user `a2ab10cde2d74c95b11a1795cc327790`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:36.107870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:36.149474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5fde1-6f19-4219-a5fd-437ead41effa admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.218330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1019/2031] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:36.263166 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.263828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.264118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.264416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.265063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.267124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.285210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:36.347797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcce731-ff3b-46ba-8f19-1da3dd66282b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.351445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.352093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.352377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.352652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.354716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:36.390797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:36.399936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1013/2032] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:36 2022] POST /identity/v3/users => generated 305 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:36.401216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:36.413137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569bfd-3a06-49e1-b270-963e56799757 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.418187 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.418862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.419226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.419570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.420342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.422126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.424519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.426066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.433320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1014/2033] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:36.435466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1020/2034] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:36.437419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:36.448052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b097a5-ed9b-46db-bffc-c66b23801212 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.452831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.453432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.453728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.453987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.454622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.456338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccc2af-c71d-48e7-acd2-f922f1707e8a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.470765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1015/2035] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:36.475436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:36.488150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf1be5-d4e6-40c5-aa07-d3935d90eb4c service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:36.492803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:36.493615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.494336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.500225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] PATH_INFO: `/v3/projects/4d0e03ec63c941bfa76d4a00fe79ef34/users/431cfef48ab14d9b871b6036516d3f63/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.526136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0e03ec63c941bfa76d4a00fe79ef34, user_id=431cfef48ab14d9b871b6036516d3f63, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:36.530609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:36.770369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1021/2036] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:36 2022] PUT /identity/v3/projects/4d0e03ec63c941bfa76d4a00fe79ef34/users/431cfef48ab14d9b871b6036516d3f63/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:36.788631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:36.789050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:36.789401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:36.840803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] MFA Rules not processed for user `431cfef48ab14d9b871b6036516d3f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:36.842256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:36.957441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1016/2037] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:36.964325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.031361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bcecd-7642-4e6e-bb45-b770ada8fb36 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.035889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.036493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.036774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.037081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.037700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.039596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.224823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1022/2038] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:36 2022] POST /identity/v3/projects => generated 407 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:37.234816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.246575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92de89-d931-4aed-9e09-5d9dc428842a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.305062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.305688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.305973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.306250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.306891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.308900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.449217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1017/2039] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:37 2022] POST /identity/v3/users => generated 310 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:37.456085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.466611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d2917-2fbe-4367-9970-0cf3a604df08 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.471665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.472269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.472558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.472862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.473563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.475618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.489637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1023/2040] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:37.491554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.501856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f1f9-1c2e-416a-985d-93f1d609d963 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.506576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.515905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.515905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.515905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] PATH_INFO: `/v3/projects/2c0d3fa498614ad78124a494819dcbd3/users/b8982dc2f17f41b39ad1a81f62a1d7ac/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.523504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0d3fa498614ad78124a494819dcbd3, user_id=b8982dc2f17f41b39ad1a81f62a1d7ac, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.526401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.648922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1018/2041] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:37 2022] PUT /identity/v3/projects/2c0d3fa498614ad78124a494819dcbd3/users/b8982dc2f17f41b39ad1a81f62a1d7ac/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:37.652824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.663791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd415a-180f-4ecf-b274-0650ef295722 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.719338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.719978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.720262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.720547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.721165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.722785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe6602-68e9-4db9-bae9-a273bef5044e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.733222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1024/2042] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:37.737452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:37.747669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805c128-29d5-47f0-bc52-76e5a5f78b8b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:37.753724 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:37.754386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.754670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.754945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] PATH_INFO: `/v3/projects/2c0d3fa498614ad78124a494819dcbd3/users/b8982dc2f17f41b39ad1a81f62a1d7ac/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.756856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0d3fa498614ad78124a494819dcbd3, user_id=b8982dc2f17f41b39ad1a81f62a1d7ac, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:37.759556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:37.769914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1019/2043] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:37 2022] PUT /identity/v3/projects/2c0d3fa498614ad78124a494819dcbd3/users/b8982dc2f17f41b39ad1a81f62a1d7ac/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:37.782627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:37.782912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:37.783192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:37.814864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] MFA Rules not processed for user `b8982dc2f17f41b39ad1a81f62a1d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:37.817322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:37.908734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1025/2044] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:38.124036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:38.234527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9b336-5b7a-4a04-819d-2ac394f2bb24 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:38.239239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:38.241425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:38.241906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:38.242365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:38.253426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:38.255624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:38.267925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1020/2045] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:38.708584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:39.221602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:39.372259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:39.382011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455fce9-a790-45d1-a3cf-215d6e7f2398 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:39.385760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:39.387059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:39.387059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:39.387059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:39.457454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:39.459410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:39.468641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1026/2046] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:40.482253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:40.483925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:40.497736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3314-3e00-4079-8ef7-7547b634fe6c service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:40.504094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:40.505036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:40.505527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:40.506089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:40.508503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:40.601457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:40.603246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795179b-f69c-4c3a-a651-9ca079f5c98e service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:40.614710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1021/2047] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:42.761737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:42.763328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:42.832745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:42.865316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757f89a-8ab3-4cbd-8d36-0fadcac1bb8e service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:42.871808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:42.872464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:42.877250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:42.877250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] PATH_INFO: `/v3/users/42ba8ea3d064459e8688978d1d3cd426` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:42.877250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ba8ea3d064459e8688978d1d3cd426)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:42.877250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:42.953852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:43.134481 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ba8ea3d064459e8688978d1d3cd426'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:43.267770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ba8ea3d064459e8688978d1d3cd426'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:43.270076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ba8ea3d064459e8688978d1d3cd426'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:43.272054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ba8ea3d064459e8688978d1d3cd426'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:43.285248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ba8ea3d064459e8688978d1d3cd426'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:43.313003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1027/2048] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:42 2022] DELETE /identity/v3/users/42ba8ea3d064459e8688978d1d3cd426 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:43.318893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:43.319524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:43.320044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:43.344687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:43.347039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:43.410698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1022/2049] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:43.421109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:43.422561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:43.531536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2332d-a01e-4d99-8caf-c8556f375672 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:43.550719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:43.551638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:43.552214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:43.552848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:43.566703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:43.569325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:43.582016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1028/2050] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:44.361021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:44.367177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:44.376572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054efb5-e3a6-43fe-b5fc-2513bcf4db2d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:44.381638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:44.382356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:44.382741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:44.383087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] PATH_INFO: `/v3/projects/28c1a9dd228b4a70b67da7618faf52e5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:44.384302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c1a9dd228b4a70b67da7618faf52e5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:44.386276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:44.588938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:44.607896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c1a9dd228b4a70b67da7618faf52e5', 'request_id': 'req-6ee9da95-9de5-4849-a770-891708f9cac4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:44.614096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c1a9dd228b4a70b67da7618faf52e5', 'request_id': 'req-6ee9da95-9de5-4849-a770-891708f9cac4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:44.727703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c1a9dd228b4a70b67da7618faf52e5', 'request_id': 'req-6ee9da95-9de5-4849-a770-891708f9cac4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:44.739250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] The token cache is being invalidate because project 28c1a9dd228b4a70b67da7618faf52e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:44.739543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:44.740248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:44.741826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1023/2051] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:44 2022] DELETE /identity/v3/projects/28c1a9dd228b4a70b67da7618faf52e5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:44.745963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:44.755923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a590c-9b8a-4300-9b8d-f6fc35760ab2 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:44.817066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:44.817852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:44.818139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:44.819173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] PATH_INFO: `/v3/users/b37a5acad6c94ce99a504fee59e43324` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:44.820104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37a5acad6c94ce99a504fee59e43324)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:44.822538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:45.121363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37a5acad6c94ce99a504fee59e43324'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:45.310397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37a5acad6c94ce99a504fee59e43324'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:45.311036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37a5acad6c94ce99a504fee59e43324'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:45.311638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37a5acad6c94ce99a504fee59e43324'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:45.319984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3f7c309-330c-428b-8eb3-4c982537c55d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37a5acad6c94ce99a504fee59e43324'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:45.337525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1029/2052] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:44 2022] DELETE /identity/v3/users/b37a5acad6c94ce99a504fee59e43324 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:45.605024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:45.927219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:45.929755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:45.958163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9da95-9de5-4849-a770-891708f9cac4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:46.020905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:46.024216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.024538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.024825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] PATH_INFO: `/v3/projects/b9802105d7464fac9969f3f68c32f0da` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.026317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9802105d7464fac9969f3f68c32f0da)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:46.029234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:46.348170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9802105d7464fac9969f3f68c32f0da', 'request_id': 'req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:46.369139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9802105d7464fac9969f3f68c32f0da', 'request_id': 'req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:46.656583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9802105d7464fac9969f3f68c32f0da', 'request_id': 'req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:46.672336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] The token cache is being invalidate because project b9802105d7464fac9969f3f68c32f0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:46.672751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:46.673765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-8f2dac2b-e2ef-4145-83c3-97b7bcbeda3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:46.679593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1024/2053] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:45 2022] DELETE /identity/v3/projects/b9802105d7464fac9969f3f68c32f0da => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:46.683965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.685230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.685522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.710896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:46.723788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:46.778980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1030/2054] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:46.787028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.787324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.787642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.790961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:46.792830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:46.810036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:46.811379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:46.823508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1025/2055] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:46.824555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:46.834095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f9718-f654-4288-bcfb-5f5db4749c7d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:46.838896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:46.839488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.839802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.840105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.840864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:46.841274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:46.843888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:46.856244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1026/2056] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:46.857933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b22c9b-427d-4d31-a4a4-07a0f226254f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:46.861260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:46.863501 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:46.864158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.864527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.865036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.867548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:46.871765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc9f45-152e-444c-922b-2f0b5a529693 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:46.876509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:46.877103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:46.877386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:46.877662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:46.878289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:46.880250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:46.932676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:46.933872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:46.944473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1031/2057] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:47.048091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1027/2058] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:46 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:47.049474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.062660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971eb1-d85c-4835-ab9b-23613187ccb8 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:47.125230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:47.125851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.126137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.126545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.127063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:47.129063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:47.286436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1032/2059] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:47 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:47.297026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.308067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256de2e-bf45-4b72-86f7-1a9e67831fb3 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:47.312614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:47.313208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.313491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.313765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.314410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:47.316025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:47.325931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1028/2060] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:47.329930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.340002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c6d27-eddd-4198-83dd-5988a5253ab3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:47.344865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:47.345497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.345787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.346065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.346706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:47.348389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac03c6f-0e13-42ea-ad7b-140a3e820df0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:47.359007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1033/2061] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:47.363209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.374245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd989ad4-412b-4f8b-85ab-3cf8a1dfd7a0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:47.378805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:47.379449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.379832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.380116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] PATH_INFO: `/v3/projects/c625983021aa470c9991db3af5745c70/users/d1176458674449acb185c4300c1fa57f/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.394514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c625983021aa470c9991db3af5745c70, user_id=d1176458674449acb185c4300c1fa57f, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:47.397700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:47.559707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1029/2062] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:47 2022] PUT /identity/v3/projects/c625983021aa470c9991db3af5745c70/users/d1176458674449acb185c4300c1fa57f/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:47.572757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.573044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.573318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.611081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] MFA Rules not processed for user `d1176458674449acb185c4300c1fa57f`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:47.611948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:47.686403 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1034/2063] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:47.691174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.748176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:47.759385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccb60c-6201-4563-8a08-d8ce1fb6e257 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:47.764193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:47.764805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:47.765093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:47.765375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:47.766014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:47.767942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:47.915570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1030/2064] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:47 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:47.919675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:47.939428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d1664-ec69-4ded-a82a-3d5fc0e46533 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.007574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.008202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.008491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.008768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.009487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.011406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.175736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1035/2065] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:47 2022] POST /identity/v3/users => generated 299 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:48.178490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.189079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a178e-bf49-403d-a0b5-f50ffac30035 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.194676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.195404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.195777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.196114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.196828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.198507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.213031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1031/2066] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:48.214367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.224464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8ade6-20dd-4153-ac81-a881ad174144 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.229329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.229977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.230258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.230532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] PATH_INFO: `/v3/projects/c372bab5dcf946e789ac99078a3d26fd/users/23f156286c434d3a9515bf10ac06e360/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.245702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c372bab5dcf946e789ac99078a3d26fd, user_id=23f156286c434d3a9515bf10ac06e360, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.248442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.355750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:48.362596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1036/2067] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:48 2022] PUT /identity/v3/projects/c372bab5dcf946e789ac99078a3d26fd/users/23f156286c434d3a9515bf10ac06e360/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:48.367195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.377245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6989201-0677-4250-8982-c904cfe0fc07 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.440604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.441122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.441522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.441805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.442447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.444843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33a2c6-f9fb-4726-9edc-112c861bc607 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.455375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1032/2068] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:48.459302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.469782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e94a5-4137-4c57-92c1-73b95578ed27 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.474696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.475362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.475668 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.475959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] PATH_INFO: `/v3/projects/c372bab5dcf946e789ac99078a3d26fd/users/23f156286c434d3a9515bf10ac06e360/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.478075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c372bab5dcf946e789ac99078a3d26fd, user_id=23f156286c434d3a9515bf10ac06e360, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.480808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.491934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1037/2069] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:48 2022] PUT /identity/v3/projects/c372bab5dcf946e789ac99078a3d26fd/users/23f156286c434d3a9515bf10ac06e360/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:48.503385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.503698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.503980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.532570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] MFA Rules not processed for user `23f156286c434d3a9515bf10ac06e360`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:48.533765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:48.599513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1033/2070] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:48.813270 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.816097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:48.878172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5052-f59c-4a65-8f35-c08c7ded8c2b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.882170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.882956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.883253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.883522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.892855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:48.894530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed835dcf-ec64-483b-bffd-8d52a09c4305 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:48.904752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1038/2071] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:48.921075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:48.922650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:48.989606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd8bb8-54e1-41e7-a3d5-623556bebaf0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:48.994025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:48.994692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:48.994940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:48.995216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] PATH_INFO: `/v3/users/308e1a57c6d34e409dbe13c37e778d39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:48.996732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308e1a57c6d34e409dbe13c37e778d39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:49.005276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:49.404208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308e1a57c6d34e409dbe13c37e778d39'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:50.226950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308e1a57c6d34e409dbe13c37e778d39'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:50.228445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308e1a57c6d34e409dbe13c37e778d39'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:50.229348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308e1a57c6d34e409dbe13c37e778d39'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:50.243511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308e1a57c6d34e409dbe13c37e778d39'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:50.256398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1034/2072] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:48 2022] DELETE /identity/v3/users/308e1a57c6d34e409dbe13c37e778d39 => generated 0 bytes in 1336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:50.261013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:50.261472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:50.261578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:50.280321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:50.281527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:50.341035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1039/2073] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:50.351347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:50.352875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:50.416433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21909f-95f9-4ef1-8f99-bef6d7897e76 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:50.420340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:50.420975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:50.421256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:50.421527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:50.431862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:50.433324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:50.456033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1035/2074] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:51.044586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:51.044586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:51.052728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea4f9-fbe4-4d38-921e-de0cf4dd7c5d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:51.057985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:51.058760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:51.059292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:51.059781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] PATH_INFO: `/v3/projects/3fbb9140016540c2a6c18f13f8afd430` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:51.061069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbb9140016540c2a6c18f13f8afd430)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:51.063248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:51.325474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:51.346944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbb9140016540c2a6c18f13f8afd430', 'request_id': 'req-d13bf801-c7b7-40c0-891c-8babd9a43610'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.355937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbb9140016540c2a6c18f13f8afd430', 'request_id': 'req-d13bf801-c7b7-40c0-891c-8babd9a43610'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.490153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbb9140016540c2a6c18f13f8afd430', 'request_id': 'req-d13bf801-c7b7-40c0-891c-8babd9a43610'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.501999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] The token cache is being invalidate because project 3fbb9140016540c2a6c18f13f8afd430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:51.502293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.503397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.505357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1040/2075] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:51 2022] DELETE /identity/v3/projects/3fbb9140016540c2a6c18f13f8afd430 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:51.511908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:51.522058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8f8a0-db5b-434e-94e6-fcc0be4bf06a service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:51.583253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:51.583666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:51.584315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:51.584605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:51.584715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:51.584956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] PATH_INFO: `/v3/users/55821b67b26f4dc891ed870a111457ad` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:51.586062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55821b67b26f4dc891ed870a111457ad)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:51.588014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:51.645456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bf801-c7b7-40c0-891c-8babd9a43610 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:51.649119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:51.649730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:51.650016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:51.650293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:51.652294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:51.712978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:51.714422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:51.724629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1041/2076] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:51.866501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55821b67b26f4dc891ed870a111457ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.950751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55821b67b26f4dc891ed870a111457ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.951420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55821b67b26f4dc891ed870a111457ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.952072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55821b67b26f4dc891ed870a111457ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:51.956887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:51.969942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55821b67b26f4dc891ed870a111457ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.003301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1036/2077] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:51 2022] DELETE /identity/v3/users/55821b67b26f4dc891ed870a111457ad => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:52.003466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:52.006436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:52.020656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee568-05b9-4b8e-99ad-ffa3ad49bb4f service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:52.068344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:52.087272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:52.087882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:52.088195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:52.088476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] PATH_INFO: `/v3/users/d1176458674449acb185c4300c1fa57f` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:52.090188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1176458674449acb185c4300c1fa57f)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:52.092896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:52.435123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1176458674449acb185c4300c1fa57f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.492495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:52.493854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:52.510189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3105-5eac-41d5-a419-403e25f1928b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:52.516250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:52.516801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:52.517076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:52.517345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] PATH_INFO: `/v3/projects/3441435986c342a3b98e570d273ae318` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:52.518488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3441435986c342a3b98e570d273ae318)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:52.520455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:52.650726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1176458674449acb185c4300c1fa57f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.651328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1176458674449acb185c4300c1fa57f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.651957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1176458674449acb185c4300c1fa57f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.660785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-5024afc3-a1a5-4344-aecf-2719e00a6bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1176458674449acb185c4300c1fa57f'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.679928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1042/2078] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:51 2022] DELETE /identity/v3/users/d1176458674449acb185c4300c1fa57f => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:52.684397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:52.685037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:52.685037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:52.704822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:52.706103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:52.767621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1043/2079] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:27:52.779915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3441435986c342a3b98e570d273ae318', 'request_id': 'req-d1193fc9-35ea-40ec-b0ed-c704774b27f6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.783349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:52.784775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:52.787037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3441435986c342a3b98e570d273ae318', 'request_id': 'req-d1193fc9-35ea-40ec-b0ed-c704774b27f6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.842870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f83f3-bcf0-4a02-a6d6-6ad6e331cb5b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:52.892466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:52.893139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:52.893425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:52.893706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:52.901589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3441435986c342a3b98e570d273ae318', 'request_id': 'req-d1193fc9-35ea-40ec-b0ed-c704774b27f6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.916929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] The token cache is being invalidate because project 3441435986c342a3b98e570d273ae318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:52.917319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.918036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d1193fc9-35ea-40ec-b0ed-c704774b27f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:52.919678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1037/2080] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:52 2022] DELETE /identity/v3/projects/3441435986c342a3b98e570d273ae318 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:52.924850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:52.925139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:52.925416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:52.946676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:52.948037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:52.962159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:52.963700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defcb7a-3e3f-4851-b173-9587af1564d8 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:52.973817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1044/2081] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:27:53.026430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1038/2082] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:53.034667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.034953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.035274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.061123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:53.063116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:53.072283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1045/2083] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:53.087408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.097305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1c7e1-6f49-4703-9607-e1ca72faeb93 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.102849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.107247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.123770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1039/2084] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:53.125294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.134789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b833-2b8e-435a-ba62-52da04ea480b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.139808 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.140372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.140657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.141066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.141652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.143576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.283721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1046/2085] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:53 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:53.288241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.302664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee812b-7e19-4169-98bd-656b7ca348fc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.364576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.365191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.366081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.366362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.367003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.369103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.478652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.480392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:53.492557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8dcd2-cc22-4dc7-84af-6b1bcb78d008 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.497190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.497817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.498095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.498367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] PATH_INFO: `/v3/users/431cfef48ab14d9b871b6036516d3f63` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.499487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=431cfef48ab14d9b871b6036516d3f63)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.501421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.567543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1040/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:53 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:53.577236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.587727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28764b-aab7-4296-805c-a1d6d663f1a8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.592432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.593017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.593302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.593580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.594223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.595821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.608649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1041/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:27:53.617508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.628871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65971a01-eb36-4fa9-a3e3-0618c20b5769 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.633985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.634618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.634894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.635174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.640120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.641779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.653875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1042/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:53.654763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:53.656634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:53.670628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624dedd-b5d1-471b-bbfd-9736c2fa7e46 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:53.675158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:53.676238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:53.676238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:53.676424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] PATH_INFO: `/v3/projects/c625983021aa470c9991db3af5745c70` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:53.677412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c625983021aa470c9991db3af5745c70)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:53.679807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:53.928616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431cfef48ab14d9b871b6036516d3f63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:53.942720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c625983021aa470c9991db3af5745c70', 'request_id': 'req-10c9e388-e716-4201-a8bb-eea4f60bf692'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:53.951536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c625983021aa470c9991db3af5745c70', 'request_id': 'req-10c9e388-e716-4201-a8bb-eea4f60bf692'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.044547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431cfef48ab14d9b871b6036516d3f63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.045182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431cfef48ab14d9b871b6036516d3f63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.045835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431cfef48ab14d9b871b6036516d3f63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.054846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431cfef48ab14d9b871b6036516d3f63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.073100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1047/2089] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:53 2022] DELETE /identity/v3/users/431cfef48ab14d9b871b6036516d3f63 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:54.074126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:54.075663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.144947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1095b-e67f-41ee-8c1f-53df332c53e3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:54.150785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:54.151445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.151850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.152166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] PATH_INFO: `/v3/projects/27b00afc58f04d35836eff41fee1268e/users/bb05adc30c434714bbb301b308096d40/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.166843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b00afc58f04d35836eff41fee1268e, user_id=bb05adc30c434714bbb301b308096d40, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:54.169585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:54.247834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c625983021aa470c9991db3af5745c70', 'request_id': 'req-10c9e388-e716-4201-a8bb-eea4f60bf692'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.260322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] The token cache is being invalidate because project c625983021aa470c9991db3af5745c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:54.260615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.261296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10c9e388-e716-4201-a8bb-eea4f60bf692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.262798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1043/2090] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:53 2022] DELETE /identity/v3/projects/c625983021aa470c9991db3af5745c70 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:54.264037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8540571-6401-49a1-b477-079b07d6683e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.264322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8540571-6401-49a1-b477-079b07d6683e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.264782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8540571-6401-49a1-b477-079b07d6683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.284636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c8540571-6401-49a1-b477-079b07d6683e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:54.285980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c8540571-6401-49a1-b477-079b07d6683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.293795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1044/2091] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:54.294813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8540571-6401-49a1-b477-079b07d6683e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:54.296100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c8540571-6401-49a1-b477-079b07d6683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.309445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8540571-6401-49a1-b477-079b07d6683e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:54.314071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:54.314705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.314987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.315263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] PATH_INFO: `/v3/users/23f156286c434d3a9515bf10ac06e360` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.316424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f156286c434d3a9515bf10ac06e360)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:54.319146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:54.328548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1048/2092] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:54 2022] PUT /identity/v3/projects/27b00afc58f04d35836eff41fee1268e/users/bb05adc30c434714bbb301b308096d40/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:54.336656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:54.336656 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.392709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5544974-c09b-475c-be5f-7db7c65d503e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:54.396396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:54.397068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.397356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.397628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.458252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:54.459766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089798fd-8b78-473c-9e96-2ec1c9861eb1 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:54.470647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1049/2093] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:54.471960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.475930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.475930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.509094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] MFA Rules not processed for user `bb05adc30c434714bbb301b308096d40`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:54.510367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.589042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1050/2094] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:54.593594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:54.595269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.612213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24c439-700f-4812-92e6-01f2665afc91 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:54.615380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f156286c434d3a9515bf10ac06e360'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.617529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:54.618143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.618431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.618708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.619342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:54.621317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:54.770225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f156286c434d3a9515bf10ac06e360'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.771036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f156286c434d3a9515bf10ac06e360'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.771840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f156286c434d3a9515bf10ac06e360'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.788257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1051/2095] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:54 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:54.791053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:54.794451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:54.794783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f156286c434d3a9515bf10ac06e360'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:54.812209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1045/2096] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:54 2022] DELETE /identity/v3/users/23f156286c434d3a9515bf10ac06e360 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:54.876798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624106e2-9678-4563-be72-56eecb6541c2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:54.960879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:54.961569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:54.961850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:54.962122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:54.962755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:54.964862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.007015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.013205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.024307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d628c00-9072-4226-a514-31df94454658 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.028992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.029628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.029910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.030183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] PATH_INFO: `/v3/users/347153b838034dcebf6c1f804ae47925` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.031343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=347153b838034dcebf6c1f804ae47925)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.033303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.148672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1052/2097] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:54 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:55.149715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.157358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.172591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83ac3-ff97-4974-a182-b4afc75d4d40 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.178254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.178899 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.179443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.180021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] PATH_INFO: `/v3/projects/4d0e03ec63c941bfa76d4a00fe79ef34` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.181065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0e03ec63c941bfa76d4a00fe79ef34)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.183072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.412582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0e03ec63c941bfa76d4a00fe79ef34', 'request_id': 'req-0a96b44d-712f-433b-a0c6-41e8ec438589'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.418726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347153b838034dcebf6c1f804ae47925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.420844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0e03ec63c941bfa76d4a00fe79ef34', 'request_id': 'req-0a96b44d-712f-433b-a0c6-41e8ec438589'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.540824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347153b838034dcebf6c1f804ae47925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.541539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347153b838034dcebf6c1f804ae47925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.542254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347153b838034dcebf6c1f804ae47925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.562380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347153b838034dcebf6c1f804ae47925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.579832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1046/2098] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:55 2022] DELETE /identity/v3/users/347153b838034dcebf6c1f804ae47925 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:27:55.580665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.582103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.658473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4648-0838-4a24-9104-e13666c0ee28 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.664170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.669162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.669162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.669162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.669162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.669162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.680757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1047/2099] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:55.681630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.683422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.687048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0e03ec63c941bfa76d4a00fe79ef34', 'request_id': 'req-0a96b44d-712f-433b-a0c6-41e8ec438589'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.699960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05847a9f-621a-41f4-bee5-a49de9593d88 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.700558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] The token cache is being invalidate because project 4d0e03ec63c941bfa76d4a00fe79ef34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:55.701159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.701898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a96b44d-712f-433b-a0c6-41e8ec438589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.703493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1053/2100] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:55 2022] DELETE /identity/v3/projects/4d0e03ec63c941bfa76d4a00fe79ef34 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:55.705864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.706512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.706800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.707079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] PATH_INFO: `/v3/projects/c372bab5dcf946e789ac99078a3d26fd` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.708264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c372bab5dcf946e789ac99078a3d26fd)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.708421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.708778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.709308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.710443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.729585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:55.730910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.742886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1054/2101] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:55.747087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.748770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.762410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44518dc8-c6e1-4935-bb62-3389a56894a0 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.765334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:55.769094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.769795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.770151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.770431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.771075 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.772737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.783219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1055/2102] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:55.784074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.788071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:55.800802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e84d8-7ea6-4cfd-9218-99f74d323292 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:55.806427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:55.807058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:55.807341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:55.807616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] PATH_INFO: `/v3/users/b8982dc2f17f41b39ad1a81f62a1d7ac` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:55.808841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8982dc2f17f41b39ad1a81f62a1d7ac)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:55.811234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:55.871012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c372bab5dcf946e789ac99078a3d26fd', 'request_id': 'req-1c945f5b-4914-4325-9cb4-626f117566a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.878229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c372bab5dcf946e789ac99078a3d26fd', 'request_id': 'req-1c945f5b-4914-4325-9cb4-626f117566a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.971668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c372bab5dcf946e789ac99078a3d26fd', 'request_id': 'req-1c945f5b-4914-4325-9cb4-626f117566a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.983963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] The token cache is being invalidate because project c372bab5dcf946e789ac99078a3d26fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:55.984265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.984967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:55.986511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1048/2103] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:55 2022] DELETE /identity/v3/projects/c372bab5dcf946e789ac99078a3d26fd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:55.988444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:55.989830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.049343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c945f5b-4914-4325-9cb4-626f117566a2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.054097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.055020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.055308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.055581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.057517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.103883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:56.119491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8982dc2f17f41b39ad1a81f62a1d7ac'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:56.126468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.127969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.138112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1049/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:56.141230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.141230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.152508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295064e1-61af-48e6-8568-f807c12b0c5f service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.157470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.158116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.158401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.158683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] PATH_INFO: `/v3/projects/21978ec05424410893c691528f348c2a/users/380f330a876b4ba39f3974b5d5e70fbd/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.175362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=21978ec05424410893c691528f348c2a, user_id=380f330a876b4ba39f3974b5d5e70fbd, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.178882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f782-54ad-45eb-acc7-195eddda2ecd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.233898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8982dc2f17f41b39ad1a81f62a1d7ac'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:56.234585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8982dc2f17f41b39ad1a81f62a1d7ac'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:56.235309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8982dc2f17f41b39ad1a81f62a1d7ac'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:56.244695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce021ded-8baa-4ba4-8634-ae36569b7c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8982dc2f17f41b39ad1a81f62a1d7ac'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:56.263752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1056/2105] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:55 2022] DELETE /identity/v3/users/b8982dc2f17f41b39ad1a81f62a1d7ac => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:56.265167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf41a-1f1d-44f6-a712-f8dc9a6f8cf1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.265463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf41a-1f1d-44f6-a712-f8dc9a6f8cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.265739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf41a-1f1d-44f6-a712-f8dc9a6f8cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.287738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f7bbf41a-1f1d-44f6-a712-f8dc9a6f8cf1 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:56.288827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f7bbf41a-1f1d-44f6-a712-f8dc9a6f8cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.385607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1057/2106] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:56.388707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268282-f515-415b-b60e-245d90de1540 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.389033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268282-f515-415b-b60e-245d90de1540 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.389366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268282-f515-415b-b60e-245d90de1540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.402904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1050/2107] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:56 2022] PUT /identity/v3/projects/21978ec05424410893c691528f348c2a/users/380f330a876b4ba39f3974b5d5e70fbd/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:56.413583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-32268282-f515-415b-b60e-245d90de1540 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:56.415601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-32268282-f515-415b-b60e-245d90de1540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.417144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.417144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.421278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.458398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] MFA Rules not processed for user `380f330a876b4ba39f3974b5d5e70fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:56.461464 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.532920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1058/2108] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:56.539042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268282-f515-415b-b60e-245d90de1540 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.557476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268282-f515-415b-b60e-245d90de1540 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.564160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.565157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.565440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.565711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.566369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:56.566673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.568423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.581222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1059/2109] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:27:56.595822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.597823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1051/2110] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:56.602809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.607349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:56.609359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a76a-bcc4-4ee7-9c85-85d507727d33 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.622868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.627380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.627380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.627380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.627380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.627922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fcfff-873c-4159-92e9-7b550376c19a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.629810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.632042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.632969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.633477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.633957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.634930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.638107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.734377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1052/2111] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:56.735355 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.803311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1060/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:27:56.805477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:56.828831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c41b7-c186-4765-9fe4-3894a7964e4c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.895136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9953-1cf6-4bc6-abc4-b5934b9fd4bd admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:56.901317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.901935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.902219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.902614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.903279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.905177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:56.928304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:56.928978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:56.929572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:56.929909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:56.930615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:56.932561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.019542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1061/2113] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:57.020544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.022773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.036271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988acfb-e5ea-4042-bd08-5e6aa3ebf20d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.041169 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.041813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.042105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.042381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] PATH_INFO: `/v3/projects/9e11882139da477398c7c95a5933669a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.043724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e11882139da477398c7c95a5933669a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.045657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.057173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1053/2114] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:56 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:57.061118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.062902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.075703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a0ca-64da-445a-8195-d5937fbd5184 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.080431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.081053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.081333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.081608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] PATH_INFO: `/v3/projects/2c0d3fa498614ad78124a494819dcbd3` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.082807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0d3fa498614ad78124a494819dcbd3)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.085323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.241817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0d3fa498614ad78124a494819dcbd3', 'request_id': 'req-2e565379-74fc-4026-9b5d-0d429e4da8d9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.247507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e11882139da477398c7c95a5933669a', 'request_id': 'req-89976d64-b463-4b17-bad2-a52b8ba34bec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.253426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0d3fa498614ad78124a494819dcbd3', 'request_id': 'req-2e565379-74fc-4026-9b5d-0d429e4da8d9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.255172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e11882139da477398c7c95a5933669a', 'request_id': 'req-89976d64-b463-4b17-bad2-a52b8ba34bec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.419093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0d3fa498614ad78124a494819dcbd3', 'request_id': 'req-2e565379-74fc-4026-9b5d-0d429e4da8d9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.431489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] The token cache is being invalidate because project 2c0d3fa498614ad78124a494819dcbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:57.431817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.432484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.434088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1054/2115] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:57 2022] DELETE /identity/v3/projects/2c0d3fa498614ad78124a494819dcbd3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:57.434849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.436229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.503328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e565379-74fc-4026-9b5d-0d429e4da8d9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.564597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.565214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.565494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.565769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.566581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.568046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.578852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1055/2116] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:57.579753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.581838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.594994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803a29-4dee-4229-964d-1066804d86b0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.600564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.601385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.601644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.601964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.602799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.604389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2720b6-b8e1-4907-ad54-43d21a205386 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.616068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1056/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:57.617651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.617945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.618220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.636106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e11882139da477398c7c95a5933669a', 'request_id': 'req-89976d64-b463-4b17-bad2-a52b8ba34bec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.639862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:57.641207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.648033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] The token cache is being invalidate because project 9e11882139da477398c7c95a5933669a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:57.648483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.649188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.650541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1057/2118] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:57.653525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1062/2119] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:57 2022] DELETE /identity/v3/projects/9e11882139da477398c7c95a5933669a => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:57.653664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.653756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.655089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.664671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e01b1-7432-42ed-8356-5cae535fb39f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.670075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.670268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976d64-b463-4b17-bad2-a52b8ba34bec admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.670631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.670924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.671204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.671919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.673535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.676076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.680754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.680754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.680754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] PATH_INFO: `/v3/projects/29866eae47fd4461861b2153917a6f45/users/7bf0a20134b24264ad2ad23ad4839db8/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.688517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1058/2120] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:57.692737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.697692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29866eae47fd4461861b2153917a6f45, user_id=7bf0a20134b24264ad2ad23ad4839db8, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.700596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb457985-be54-4bbb-93dd-0b218e97c306 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.706601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.721260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95037b1-a23a-42ed-90fd-00585d50c675 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.726812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.727477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.727800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.728083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] PATH_INFO: `/v3/users/a2ab10cde2d74c95b11a1795cc327790` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.729748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ab10cde2d74c95b11a1795cc327790)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.731751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:57.816551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1063/2121] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:57 2022] PUT /identity/v3/projects/29866eae47fd4461861b2153917a6f45/users/7bf0a20134b24264ad2ad23ad4839db8/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:57.817848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.818170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.818626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.839752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:57.840999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.907296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1064/2122] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:57.908202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:57.909584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:57.924943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e479-a1a8-4c8b-9bb0-36a2f531b40a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:57.935370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:57.941521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:57.941521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:57.941521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] PATH_INFO: `/v3/projects/0257b42e6be04e84b906479efc30b3b8/users/fadf0d9cf4cf4c06bb153fe61736c17c/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:57.953663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ab10cde2d74c95b11a1795cc327790'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:57.956083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0257b42e6be04e84b906479efc30b3b8, user_id=fadf0d9cf4cf4c06bb153fe61736c17c, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:57.959520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.027707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ab10cde2d74c95b11a1795cc327790'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.028791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ab10cde2d74c95b11a1795cc327790'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.029953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ab10cde2d74c95b11a1795cc327790'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.034471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1065/2123] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:57 2022] PUT /identity/v3/projects/0257b42e6be04e84b906479efc30b3b8/users/fadf0d9cf4cf4c06bb153fe61736c17c/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:58.035054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.038105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.051315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ab10cde2d74c95b11a1795cc327790'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.068587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1059/2124] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:57 2022] DELETE /identity/v3/users/a2ab10cde2d74c95b11a1795cc327790 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:58.072535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.076969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.147227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157ce00-284e-4735-ab09-40beba7de5f4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.181570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53420-620f-4936-a944-d2694d13fafd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.187535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.188343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.188543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.188819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.189753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:58.191417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.207536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1060/2125] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:58.207536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.210500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.229639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948874-e834-4c6b-b59d-52b3762f2e61 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.235889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.236629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.237024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.237536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.238831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:27:58.239253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:58.241194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41447e0-d898-42b4-ada7-8548a5b651f8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.256711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1061/2126] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:27:58.258598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.259065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.259413 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.262210 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.263024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.263422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.263892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] PATH_INFO: `/v3/users/4e1bc3f6ea7f4c898bd43f22e3b2bac1` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.265387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1bc3f6ea7f4c898bd43f22e3b2bac1)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:58.267487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.301072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] MFA Rules not processed for user `fadf0d9cf4cf4c06bb153fe61736c17c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:58.304024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.381723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1062/2127] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:58.385677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.393942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756fb7-447e-404c-beec-e3e351e3f241 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.400856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.401753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.402120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.402459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] PATH_INFO: `/v3/projects/29866eae47fd4461861b2153917a6f45/users/7bf0a20134b24264ad2ad23ad4839db8/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.404566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29866eae47fd4461861b2153917a6f45, user_id=7bf0a20134b24264ad2ad23ad4839db8, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:58.407482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.429634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1063/2128] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:58 2022] PUT /identity/v3/projects/29866eae47fd4461861b2153917a6f45/users/7bf0a20134b24264ad2ad23ad4839db8/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:58.429634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.510655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c7393-9d28-45f1-87b6-6540974ea343 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.516102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.517274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.517274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.517512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.518113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:58.520079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:58.597651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1bc3f6ea7f4c898bd43f22e3b2bac1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.622299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:58.685879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1064/2129] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:27:58 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:58.687125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.689912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.711855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1bc3f6ea7f4c898bd43f22e3b2bac1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.712938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1bc3f6ea7f4c898bd43f22e3b2bac1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.714669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1bc3f6ea7f4c898bd43f22e3b2bac1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.728598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-b87ca385-f391-48e1-879b-647d2d59373c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1bc3f6ea7f4c898bd43f22e3b2bac1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:58.766208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1066/2130] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:58 2022] DELETE /identity/v3/users/4e1bc3f6ea7f4c898bd43f22e3b2bac1 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:58.770502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.770795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.771077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.790422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4345a4a-5d31-4b9d-a6fe-22cb7f3cc255 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:58.813081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] MFA Rules not processed for user `7bf0a20134b24264ad2ad23ad4839db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:58.814526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.843749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:27:58.895222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:58.896288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:58.897549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:58.897980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:58.901587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:58.970695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1067/2131] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:58.972760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:58.975292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.022940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.025259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.038712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1065/2132] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:59.039971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.041531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.086182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31453508-81fb-43ab-8e37-6786faf957c6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.096708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.097598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.097886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.098217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] PATH_INFO: `/v3/projects/830b9996c1714040840268f88436fa00` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.100795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830b9996c1714040840268f88436fa00)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.104195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.149334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab73-26f1-4ab6-9aeb-ffa593a7e478 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.155077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.155883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.156238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.156891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.157628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.161386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d1217-65ee-4bfe-8d9d-3ad73d15bb16 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.231635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830b9996c1714040840268f88436fa00', 'request_id': 'req-f9b1023f-1336-4c99-81ba-18427c624140'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.238883 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830b9996c1714040840268f88436fa00', 'request_id': 'req-f9b1023f-1336-4c99-81ba-18427c624140'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.289228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1066/2133] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:27:59 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:27:59.289228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.289228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.289228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.307970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:59.309174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.372218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1067/2134] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:59.372860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.374404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.376435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830b9996c1714040840268f88436fa00', 'request_id': 'req-f9b1023f-1336-4c99-81ba-18427c624140'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.390174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] The token cache is being invalidate because project 830b9996c1714040840268f88436fa00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:27:59.390558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.391328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.393278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1068/2135] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:27:58 2022] DELETE /identity/v3/projects/830b9996c1714040840268f88436fa00 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:27:59.394166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.395490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.409139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1023f-1336-4c99-81ba-18427c624140 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.414063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.414745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.415039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.415315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] PATH_INFO: `/v3/users/fadf0d9cf4cf4c06bb153fe61736c17c` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.416502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadf0d9cf4cf4c06bb153fe61736c17c)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.418431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.438591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3611d3-6ada-45df-9f71-a6e1fb43bd15 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.442500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.443108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.444075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.444349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.446132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.508263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.509761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.519086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1068/2136] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:59.519927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.522162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.536683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605077-4821-4240-8fb7-b9fc5b7f3bd6 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.541924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.542595 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.542938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.543229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.545110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.547694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.558947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1069/2137] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:59.560885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.563522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.627108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f2f5-32fe-4261-9e63-a3133202bedb admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.634772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.635389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.635713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.635966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.638009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.652005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.653702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647550fd-fc0e-47b5-a178-3ee3e60ef9ca service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.664546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1070/2138] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:27:59.665913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.666213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.666493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.695179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:59.696509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.706686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1071/2139] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:59.707407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.723533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1107b1-33ec-445d-ba5a-38cece7bd3e8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.729689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.730328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.730614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.730894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.731566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.733396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e86f5-0dcf-4496-82ad-e51c0ef26945 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.749087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1072/2140] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:27:59.761362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.764621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.765032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.765723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadf0d9cf4cf4c06bb153fe61736c17c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.790566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:27:59.792019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:27:59.805295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1073/2141] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:27:59.806141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:27:59.816245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14e89e-f10e-4f3f-832e-df6fae5c0791 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:27:59.821338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:27:59.828244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:27:59.828244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:27:59.828244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] PATH_INFO: `/v3/projects/25f601a4b4184d858fa699a0ab0bf6b4/users/35b1f88d727a4ab191c6602c2c3d06c9/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:27:59.839066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f601a4b4184d858fa699a0ab0bf6b4, user_id=35b1f88d727a4ab191c6602c2c3d06c9, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:27:59.841777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53c0fb-23ca-48aa-baaf-2db7b0fc3307 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:27:59.931736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadf0d9cf4cf4c06bb153fe61736c17c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.936122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadf0d9cf4cf4c06bb153fe61736c17c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.936953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadf0d9cf4cf4c06bb153fe61736c17c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.949258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadf0d9cf4cf4c06bb153fe61736c17c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:27:59.999801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1069/2142] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:27:59 2022] DELETE /identity/v3/users/fadf0d9cf4cf4c06bb153fe61736c17c => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:00.001097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.004642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.028598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1074/2143] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:27:59 2022] PUT /identity/v3/projects/25f601a4b4184d858fa699a0ab0bf6b4/users/35b1f88d727a4ab191c6602c2c3d06c9/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:00.043563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344be9e-7371-4d43-869a-c6f97106cf83 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.043563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344be9e-7371-4d43-869a-c6f97106cf83 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.043563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344be9e-7371-4d43-869a-c6f97106cf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.065526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d344be9e-7371-4d43-869a-c6f97106cf83 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:00.080879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d344be9e-7371-4d43-869a-c6f97106cf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.134745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9357e-76b6-418e-a423-755a3a68552a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.167074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1075/2144] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:00.168600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.168975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.169334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.199162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] MFA Rules not processed for user `35b1f88d727a4ab191c6602c2c3d06c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:00.199768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.200390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.200676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.200861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.200985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.201584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:00.201894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.203831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.217502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1070/2145] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:00.218214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.220350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.270362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1076/2146] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:00.272459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.282989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90614-bef7-48cf-a391-ed1a1cfcf261 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.284248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367458-d7a6-4ba7-a489-262f01a3b8a5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.288595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.288902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.289207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.289490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.289579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.289767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.289865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.290063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.290694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.292688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.299642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.301428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.312703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1071/2147] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:00.313536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.315117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.333630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa099-81a0-4f23-9242-e8defd18b634 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.340119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.340994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.341408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.341818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.342823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.345187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.471533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1077/2148] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/projects => generated 389 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:00.471533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.471533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.533792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb82a-f86c-46cb-babe-8a1123844e36 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.539341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.540053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.540332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.540623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] PATH_INFO: `/v3/projects/062fe0bdb0894b4cab318f203d39534d` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.541813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=062fe0bdb0894b4cab318f203d39534d)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.543751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.577608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1072/2149] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/projects => generated 379 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:00.578428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.647767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b807ea6-631e-451e-b3df-c49fe0bdffe2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.652860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.653584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.653853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.654130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.655025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.656985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.722149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062fe0bdb0894b4cab318f203d39534d', 'request_id': 'req-2edf77b2-d367-4a18-ab99-8a965df83dfe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:00.729620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062fe0bdb0894b4cab318f203d39534d', 'request_id': 'req-2edf77b2-d367-4a18-ab99-8a965df83dfe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:00.768382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:00.807029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1073/2150] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:00.807995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.818880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03636-9b2b-400a-acaa-7eb4756c1ad2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.850234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062fe0bdb0894b4cab318f203d39534d', 'request_id': 'req-2edf77b2-d367-4a18-ab99-8a965df83dfe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:00.871279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] The token cache is being invalidate because project 062fe0bdb0894b4cab318f203d39534d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:00.871620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:00.872436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:00.875698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1078/2151] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:00 2022] DELETE /identity/v3/projects/062fe0bdb0894b4cab318f203d39534d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:00.875698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.876154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.898666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.899416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.899741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.900129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.900944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.902905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.953260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf77b2-d367-4a18-ab99-8a965df83dfe admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.958458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.959086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.959390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.959659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.960364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.966691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:00.972140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1079/2152] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:00.972949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:00.974936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:00.988325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908a6a-e4b1-4020-96a9-963d0cee8b37 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:00.993379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:00.994088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:00.994248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:00.994552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] PATH_INFO: `/v3/users/df89414a7e95448197dd77b360cf18bb` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:00.995749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df89414a7e95448197dd77b360cf18bb)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:00.997650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.057525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1074/2153] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:00 2022] POST /identity/v3/users => generated 290 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:01.058434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.069252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf8da-00ee-4e75-83d5-0de786a5e108 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.074805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.075404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.075781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.076061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.076741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.078340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.090099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1075/2154] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:01.091093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.092788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.105301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbbacf-9707-4d05-9395-5442571b68a3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.109997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.110609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.110891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.111166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] PATH_INFO: `/v3/projects/0257b42e6be04e84b906479efc30b3b8` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.112408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0257b42e6be04e84b906479efc30b3b8)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.114296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.310683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0257b42e6be04e84b906479efc30b3b8', 'request_id': 'req-85d1dd30-acdd-4fde-a4b5-b63997927af3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.312839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df89414a7e95448197dd77b360cf18bb'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.317956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0257b42e6be04e84b906479efc30b3b8', 'request_id': 'req-85d1dd30-acdd-4fde-a4b5-b63997927af3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.484701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df89414a7e95448197dd77b360cf18bb'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.485415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df89414a7e95448197dd77b360cf18bb'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.486072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df89414a7e95448197dd77b360cf18bb'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.505809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df89414a7e95448197dd77b360cf18bb'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.519101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1080/2155] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:00 2022] DELETE /identity/v3/users/df89414a7e95448197dd77b360cf18bb => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:01.520157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.523606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.623247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cedc28-2947-4804-8465-6d0d0edc1fd2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.629395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.630212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.630677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.631081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.632325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.633494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.650791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1081/2156] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:01.650791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.652663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.668611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8407-d022-4bc7-916e-2f465abd8604 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.674113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0257b42e6be04e84b906479efc30b3b8', 'request_id': 'req-85d1dd30-acdd-4fde-a4b5-b63997927af3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.675264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.675967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.676253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.676531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] PATH_INFO: `/v3/projects/92494f2ed3104d41b5a84bdc2ddf3ebb/users/df2a2ca41653426a89aded6986e94247/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.678822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:01.692232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] The token cache is being invalidate because project 0257b42e6be04e84b906479efc30b3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:01.692556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.693217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:01.694814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1076/2157] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:01 2022] DELETE /identity/v3/projects/0257b42e6be04e84b906479efc30b3b8 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:01.695861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.697313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.699875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92494f2ed3104d41b5a84bdc2ddf3ebb, user_id=df2a2ca41653426a89aded6986e94247, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.703310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b563-5111-4f19-8795-158cbc68e914 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.783465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1dd30-acdd-4fde-a4b5-b63997927af3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.790421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.791285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.791575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.791880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.792529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.794144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b2bf-0dde-49e0-a5b9-5d48c4a9d7f4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:01.805799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1077/2158] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:01.806978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.807268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.807544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.832595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:01.833805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.843273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1078/2159] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:01.844190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:01.869691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bd9a9-0ee7-4972-9563-24cd9329eef8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:01.885457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1082/2160] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:01 2022] PUT /identity/v3/projects/92494f2ed3104d41b5a84bdc2ddf3ebb/users/df2a2ca41653426a89aded6986e94247/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:01.892560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.893025 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.893490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.935358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:01.936880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:01.961447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:01.962123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:01.962399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:01.962743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] PATH_INFO: `/v3/projects/150c88f4277f41bb87d459d6d7c2d3d5/users/879dab227b9c4e5a84cc2ee920476527/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:01.982202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150c88f4277f41bb87d459d6d7c2d3d5, user_id=879dab227b9c4e5a84cc2ee920476527, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:01.984967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a84e48-8682-40f6-9a46-7d46ebd791f8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.021049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1083/2161] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:02.022605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.024303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.083499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1079/2162] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:01 2022] PUT /identity/v3/projects/150c88f4277f41bb87d459d6d7c2d3d5/users/879dab227b9c4e5a84cc2ee920476527/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:02.085964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.086472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.086907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.126042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060ba1-e671-4e8b-b946-e5635c8b2adc None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.129974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] MFA Rules not processed for user `df2a2ca41653426a89aded6986e94247`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:02.132221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.181953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.182744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.183038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.183334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.185850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.208736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1080/2163] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:02.210466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.270514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.272685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52334c-0277-41b6-ade9-65655d76a33d service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.283522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1084/2164] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:02.284886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.285186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.285551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.304398 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d7c31-aee2-4766-9fed-3b5cda929727 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.311684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.312499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.312770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.313199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.313939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:02.314659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.316260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.318212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] MFA Rules not processed for user `879dab227b9c4e5a84cc2ee920476527`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:02.320065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.331962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1081/2165] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:02.332950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.335375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.350810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d3dd4-2e85-4eb5-840e-0923828dd4d6 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.355866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.356486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.356788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.357389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.368414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.369859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.385728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1082/2166] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:02.386736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.388717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.407405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538728f-4aa3-4984-9675-c135a5d34b63 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.412902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.415702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1085/2167] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:02.415702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.415962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.415962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.415962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] PATH_INFO: `/v3/projects/43295fd82b56469c8a4957dca2855478` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.416193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43295fd82b56469c8a4957dca2855478)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.418411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.427086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e11b64-184e-449f-b9f8-df4126610c6b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.433163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.433865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.434157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.434555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.435213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.437996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.560978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43295fd82b56469c8a4957dca2855478', 'request_id': 'req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:02.567950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43295fd82b56469c8a4957dca2855478', 'request_id': 'req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:02.629282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1086/2168] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:02.630172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.679029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43295fd82b56469c8a4957dca2855478', 'request_id': 'req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:02.691855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] The token cache is being invalidate because project 43295fd82b56469c8a4957dca2855478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:02.692152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:02.694195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:02.696637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1083/2169] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:02 2022] DELETE /identity/v3/projects/43295fd82b56469c8a4957dca2855478 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:02.697409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.699673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.707346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bd10b-92ca-4b15-a86a-7c054af1d739 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.713018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.713620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.713969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.714314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.714969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.716903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.768152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23122c-1a2d-4c9e-b7b7-f075ac315ea8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.772991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.773597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.773876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.774188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.774981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.776932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:02.874081 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1087/2170] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:02.875155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.878850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.919796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1084/2171] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:02.920496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:02.924703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:02.938541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8548f-0af5-4b20-99d0-42c05032e3bc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.976299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611d9b-422b-4597-bb3b-d20b56989143 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:02.981372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:02.982044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:02.982356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:02.982636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] PATH_INFO: `/v3/users/b22eae5a0b1d4bb4ab1d5a89837e454a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:02.983983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22eae5a0b1d4bb4ab1d5a89837e454a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:02.985814 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.000993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.001608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.001893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.002268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.003355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.006327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.417682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1085/2172] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:02 2022] POST /identity/v3/users => generated 295 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:03.418655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.428458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bdac-218b-4f51-a49a-bf9d0ab3194f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.433281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.433869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.434152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.434428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.435064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.436687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.447696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1086/2173] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:03.448354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.459571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74c49-e59c-4f91-bf62-237b339c60ef admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.464462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.465069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.465396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.465698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.466340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.468079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.479306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1087/2174] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:03.480181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.489797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd43724-837d-44ce-a59f-59ea6fecd153 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.494448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.495024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.495310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.495581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.496250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.497823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.511074 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1088/2175] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:03.511783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.520455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22eae5a0b1d4bb4ab1d5a89837e454a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:03.522054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f038a0a-06da-4dba-9e4a-30f70e30e9c1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.526688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.527329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.527609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.527914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] PATH_INFO: `/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533/users/9408eb0a36234f6a90eb16dec190059d/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.543660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c237ec7a3c45b6b6d574fd428a5533, user_id=9408eb0a36234f6a90eb16dec190059d, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.547787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.678663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22eae5a0b1d4bb4ab1d5a89837e454a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:03.684707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22eae5a0b1d4bb4ab1d5a89837e454a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:03.684707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22eae5a0b1d4bb4ab1d5a89837e454a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:03.695701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22eae5a0b1d4bb4ab1d5a89837e454a'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:03.709942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1088/2176] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:02 2022] DELETE /identity/v3/users/b22eae5a0b1d4bb4ab1d5a89837e454a => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:03.710761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.713982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:03.772483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:03.778602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1089/2177] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:03 2022] PUT /identity/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533/users/9408eb0a36234f6a90eb16dec190059d/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:03.782971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.785479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:03.797431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45348-db77-4d87-ae25-1e22f11bc520 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.875026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.875720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.876004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.876279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] PATH_INFO: `/v3/projects/cb122d1d37094659be28ec1533e73aa4/users/0111cd29f2774178834b86f35a777250/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.885119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda426ff-7c82-4c20-9729-162f8149ec2c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.890717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.891343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.891680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.891931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.892588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.894252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.897061 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb122d1d37094659be28ec1533e73aa4, user_id=0111cd29f2774178834b86f35a777250, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.900101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024867dc-bb4c-442d-ad1b-884245d02651 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.905686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1090/2178] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:03.909999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:03.921063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb287e-1ec8-417c-bf8b-c36c3f92ba4f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:03.926623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:03.928458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.928458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.928811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] PATH_INFO: `/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533/users/9408eb0a36234f6a90eb16dec190059d/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:03.931311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c237ec7a3c45b6b6d574fd428a5533, user_id=9408eb0a36234f6a90eb16dec190059d, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:03.934087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ea57-5d4a-4c7d-9515-a9ceded7b57f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:03.945628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1091/2179] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:03 2022] PUT /identity/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533/users/9408eb0a36234f6a90eb16dec190059d/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:03.956010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:03.956315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:03.956623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.008895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] MFA Rules not processed for user `9408eb0a36234f6a90eb16dec190059d`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:04.010462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:04.026407 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1089/2180] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:03 2022] PUT /identity/v3/projects/cb122d1d37094659be28ec1533e73aa4/users/0111cd29f2774178834b86f35a777250/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:04.033988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.034281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.034565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.088127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] MFA Rules not processed for user `0111cd29f2774178834b86f35a777250`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:04.090162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:04.131562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1092/2181] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:04.142437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:04.144683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:04.198517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1090/2182] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:04.234361 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e90c7-9f4d-447a-8b92-1e7e4b959017 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:04.296590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:04.297487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.297972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.298446 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.364140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:04.366483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:04.379145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1093/2183] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:04.506551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:04.508098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:04.581871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecc26d-5131-4d4b-9501-1e0e3c4893ab None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:04.586914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:04.587641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.588189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.589254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] PATH_INFO: `/v3/projects/cffe2374b2e94655a9d76227805c3b31` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.590251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffe2374b2e94655a9d76227805c3b31)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:04.592293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:04.613739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:04.625453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b624e1-8c29-4c23-92a1-a3c1ad99a7f1 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:04.629303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:04.629917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.630223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.630507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.639566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:04.641045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:04.651299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1094/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:04.773961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffe2374b2e94655a9d76227805c3b31', 'request_id': 'req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:04.781375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffe2374b2e94655a9d76227805c3b31', 'request_id': 'req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:04.918583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:04.919990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:04.932055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e09f5-b826-4016-a345-1d7a750b074c service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:04.947450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffe2374b2e94655a9d76227805c3b31', 'request_id': 'req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:04.974918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] The token cache is being invalidate because project cffe2374b2e94655a9d76227805c3b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:04.975515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:04.976486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c45b9c73-9a2f-4cc1-862b-5fbcbdacebd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:04.978271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1091/2185] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:04 2022] DELETE /identity/v3/projects/cffe2374b2e94655a9d76227805c3b31 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:04.984649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974851-364b-49b8-a256-2507ed1368d0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.984649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974851-364b-49b8-a256-2507ed1368d0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.984908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974851-364b-49b8-a256-2507ed1368d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.994762 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:04.995380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:04.995811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:04.996219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:04.998170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:05.017104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ae974851-364b-49b8-a256-2507ed1368d0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:05.018343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ae974851-364b-49b8-a256-2507ed1368d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:05.083144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.085269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.105253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1092/2186] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:05.105850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1095/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:05.108422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.108831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.112996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.141938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:05.141938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:05.152907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1093/2188] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:05.157836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.172692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56386e-d96d-4fa6-82c1-988fd3d1d00a service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.181212 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.181872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.182161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.182440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.183394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:05.183746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.185870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.204375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1096/2189] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:05.206315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.222753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ae46f-b3cd-4e58-99b4-29389ad28fd6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.228613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.229168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.229458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.229739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.230409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.232861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.497153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1094/2190] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:05 2022] POST /identity/v3/projects => generated 375 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:05.501439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.511416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4859-7b14-4f15-ab2c-a804cd1e05d7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.585220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.585220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.585220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.585220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.585659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.587381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.756590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1097/2191] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:05 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:05.758408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.777842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2731fa-30d8-428c-92a9-216e213caeee admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.782974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.783442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.783757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.784034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.784688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.786284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.801195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1095/2192] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:05.802586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.823711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67e222-f0cd-4b87-9b6c-5ed3fd7444a9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.830294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.831518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.831930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.832356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.833385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.837972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409109c-538c-4c4b-b912-0096a58d7251 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:05.855677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1098/2193] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:05.856447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:05.870153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76147-6696-464a-b069-75a254c1d8b8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:05.873746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:05.881571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:05.881571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:05.881571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:05.881571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] PATH_INFO: `/v3/projects/7f0f6d5fc07746908fea709b8c261521/users/aa31273cf7a04f4287b5545c89176bfc/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:05.900825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0f6d5fc07746908fea709b8c261521, user_id=aa31273cf7a04f4287b5545c89176bfc, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:05.904224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:06.016076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1096/2194] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:05 2022] PUT /identity/v3/projects/7f0f6d5fc07746908fea709b8c261521/users/aa31273cf7a04f4287b5545c89176bfc/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:06.018144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:06.018721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:06.019205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:06.084445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] MFA Rules not processed for user `aa31273cf7a04f4287b5545c89176bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:06.086313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:06.165671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1099/2195] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:06.182220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:06.184704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:06.208666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:06.295585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f3bd-7844-4377-84c4-5b87ba8a6605 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:06.299533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:06.300281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:06.300725 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:06.301095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:06.310560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:06.313058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:06.322965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1097/2196] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:07.276721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:07.278865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:07.339939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760639-b583-4427-a5f8-45ebf982e034 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:07.343799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:07.344443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:07.344725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:07.345028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:07.346764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:07.420099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:07.421854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:07.433670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1100/2197] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:08.077873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:08.610264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:08.750851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:08.750851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:08.824803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b44fc-740f-47e6-ab34-cca5612ee730 service glance] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:08.829517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:08.830161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:08.830760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:08.831138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] PATH_INFO: `/v3/users/1ab4dbb6c98f4c99bbb06f586aaddeba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:08.832311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab4dbb6c98f4c99bbb06f586aaddeba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:08.834817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:09.159508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab4dbb6c98f4c99bbb06f586aaddeba'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:09.348485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab4dbb6c98f4c99bbb06f586aaddeba'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:09.349389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab4dbb6c98f4c99bbb06f586aaddeba'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:09.350028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab4dbb6c98f4c99bbb06f586aaddeba'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:09.364825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-01568f74-1679-4884-a5ca-a6b6e3d966df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab4dbb6c98f4c99bbb06f586aaddeba'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:09.377834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1098/2198] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:08 2022] DELETE /identity/v3/users/1ab4dbb6c98f4c99bbb06f586aaddeba => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:09.754799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:09.766508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fee3e-d3cb-42a1-b060-110e0db5ca9c service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:09.838180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:09.838916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:09.839203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:09.840366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] PATH_INFO: `/v3/projects/4a866fb1264140c1a1056696bf6fabc7` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:09.840972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a866fb1264140c1a1056696bf6fabc7)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:09.842911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.113629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a866fb1264140c1a1056696bf6fabc7', 'request_id': 'req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:10.120577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a866fb1264140c1a1056696bf6fabc7', 'request_id': 'req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:10.284216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a866fb1264140c1a1056696bf6fabc7', 'request_id': 'req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:10.297466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] The token cache is being invalidate because project 4a866fb1264140c1a1056696bf6fabc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:10.297799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:10.298509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d1aa76aa-01d0-4d7f-ad5e-962f919b5fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:10.300647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1101/2199] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:09 2022] DELETE /identity/v3/projects/4a866fb1264140c1a1056696bf6fabc7 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:10.305792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.306261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.306406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.329326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:10.330414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:10.394129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1099/2200] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:10.402971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.403371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.403768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.424440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:10.425639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:10.434355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1102/2201] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:10.438450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.448912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33edde-c2a1-4d7a-9095-d236d36f4f24 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:10.453596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:10.454200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.454479 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.454911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.455527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:10.455948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:10.457552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.471560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1100/2202] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:28:10.476810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.486454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436659d-2f5e-40fc-8f30-1427924f4f77 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:10.491555 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:10.491996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.492281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.492558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.493188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:10.495156 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.689416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1103/2203] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:10 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:10.694633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.705888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260fdc-52ec-4e09-ac86-644b8808b98d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:10.768465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:10.769245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.769534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.769938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.770689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:10.773860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.889959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.891847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:10.893640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1101/2204] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:10 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:10.901312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.907280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622635a-8660-43c9-a21a-11adb483ba4f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:10.914296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:10.915259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.915767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.916195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] PATH_INFO: `/v3/users/0111cd29f2774178834b86f35a777250` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.917767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0111cd29f2774178834b86f35a777250)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:10.919964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.948826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b96e7-fc24-4678-90dd-ad98c44db13d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:10.948826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:10.948826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:10.948826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:10.949982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:10.949982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:10.950342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:10.973139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1102/2205] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:10.984927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:10.987231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:11.016162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae62fc2-4a53-45af-8807-cde14ca4cf55 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:11.028661 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:11.029662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.030141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.030606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.031888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:11.035732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:11.065560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1103/2206] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:11.066945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:11.082301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf56e-c53d-420d-bbd9-a51a60cfd0b4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:11.090145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:11.091085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.091367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.091672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] PATH_INFO: `/v3/projects/340975b28fc242b5bf4a3364fe702ad4/users/cfbdd49e2fa64b428d957d9dcf1d6846/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.120334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340975b28fc242b5bf4a3364fe702ad4, user_id=cfbdd49e2fa64b428d957d9dcf1d6846, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:11.124042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7217c-d1ca-4fac-b4ec-747c630bd466 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:11.306192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1104/2207] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:11 2022] PUT /identity/v3/projects/340975b28fc242b5bf4a3364fe702ad4/users/cfbdd49e2fa64b428d957d9dcf1d6846/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:11.313215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.313759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.314044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.335599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:11.336941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:11.355752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0111cd29f2774178834b86f35a777250'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:11.406486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1105/2208] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:11.420600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:11.422127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:11.484854 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee22c-5431-4c09-bce4-0115efe552c4 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:11.489254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:11.489876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.490188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.490470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.502249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:11.504329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0329fe4-517d-4fdf-a174-006debad5110 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:11.517941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1106/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:11.539019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0111cd29f2774178834b86f35a777250'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:11.539909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0111cd29f2774178834b86f35a777250'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:11.542314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0111cd29f2774178834b86f35a777250'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:11.554908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0111cd29f2774178834b86f35a777250'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:11.578557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1104/2210] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:10 2022] DELETE /identity/v3/users/0111cd29f2774178834b86f35a777250 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:11.583311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c86040-86cc-40aa-98df-153c278546ef None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.584246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c86040-86cc-40aa-98df-153c278546ef None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.584535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c86040-86cc-40aa-98df-153c278546ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.604733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-33c86040-86cc-40aa-98df-153c278546ef None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:11.606126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-33c86040-86cc-40aa-98df-153c278546ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:11.667235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1107/2211] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:11.678401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:11.680351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:11.742534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:11.752396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea374-2b4a-431b-932b-f128c8a0f30a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:11.756109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:11.756748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:11.757053 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:11.757444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:11.766664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:11.768174 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb9f4-4cd5-4462-90a4-0e264f8b0f3c service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:11.780538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1105/2212] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:12.434153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c86040-86cc-40aa-98df-153c278546ef None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:12.435794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-33c86040-86cc-40aa-98df-153c278546ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:12.450178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c86040-86cc-40aa-98df-153c278546ef None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:12.457110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:12.458085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:12.459983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:12.460290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] PATH_INFO: `/v3/projects/cb122d1d37094659be28ec1533e73aa4` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:12.468145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb122d1d37094659be28ec1533e73aa4)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:12.470209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:12.742159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb122d1d37094659be28ec1533e73aa4', 'request_id': 'req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:12.749105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb122d1d37094659be28ec1533e73aa4', 'request_id': 'req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:12.993330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb122d1d37094659be28ec1533e73aa4', 'request_id': 'req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:12.997566 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:13.009094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] The token cache is being invalidate because project cb122d1d37094659be28ec1533e73aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:13.009491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:13.010146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12b2b0cb-3e52-4138-828b-2c9b274f3e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:13.011733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1108/2213] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:12 2022] DELETE /identity/v3/projects/cb122d1d37094659be28ec1533e73aa4 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:13.016495 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.017087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.017087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.037289 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:13.038609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:13.098444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1106/2214] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:13.107745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.107913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.108161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.129422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:13.130526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:13.142457 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1109/2215] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:13.146844 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.162428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27558-2f18-411d-ad9a-923e6cd180e3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.168193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.169119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.169559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.170039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.170940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:13.171518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.173615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:13.189851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1107/2216] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:13.192813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.212500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bde0f-0f40-432f-9cc9-5e35187b61e7 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.217675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.218283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.223452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.223452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.223452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.223452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:13.409229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1110/2217] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:13 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:13.413847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.423768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc7a85-220d-4435-966f-09a57d8bf97c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.480573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.481182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.481474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.481750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.482395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.484441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:13.757341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1108/2218] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:13 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:13.759762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.770022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bca0c-ce7a-4b12-be45-24943438e13a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.776165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.776846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.777217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.777586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.778300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.779967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:13.791160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1111/2219] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:13.795526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.805884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feda51-773b-4bf0-b45c-114b8e220a72 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.810566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.811185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.811471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.811779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.812434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.814060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d17854-3876-498b-bb97-0136d9ff6c63 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:13.828817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1109/2220] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:13.830102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:13.842085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79474352-e07c-4943-add5-68995594cac5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:13.846962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:13.847623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:13.847938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:13.848219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] PATH_INFO: `/v3/projects/0f40505192a74e1793f7562e6b8e7fc7/users/7a76979b6ce24ebc9c0bea8d3e0a150b/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:13.863839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f40505192a74e1793f7562e6b8e7fc7, user_id=7a76979b6ce24ebc9c0bea8d3e0a150b, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:13.866488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:14.064246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1112/2221] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:13 2022] PUT /identity/v3/projects/0f40505192a74e1793f7562e6b8e7fc7/users/7a76979b6ce24ebc9c0bea8d3e0a150b/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:14.073579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.074125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.074319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.101317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:14.133456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] MFA Rules not processed for user `7a76979b6ce24ebc9c0bea8d3e0a150b`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:14.136630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:14.205812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1110/2222] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:14.211068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:14.288864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6536456-feeb-4d17-8900-f2f17d07d0bd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:14.294014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:14.294699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.295145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.295368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.298138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:14.299628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:14.528195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1113/2223] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:14 2022] POST /identity/v3/projects => generated 377 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:14.532043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:14.543836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e8190-db23-44a5-bd69-28450dc1a1e9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:14.603095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:14.603856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.604282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.604655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.605608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:14.607763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:14.751564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1111/2224] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:14 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:14.755261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:14.766819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0367c3-b1e3-447f-8413-fc51099761a2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:14.774052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:14.774754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.775096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.775437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.776876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:14.778504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:14.792659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1114/2225] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:14.797646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:14.809534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6985-5a69-4a09-ae73-3761d6519475 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:14.814961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:14.815599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.816000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.816337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.817051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:14.819574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577dbe0-8b56-49df-b31e-abc4166dc550 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:14.833726 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1112/2226] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:14.835635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:14.846000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a0713-54f8-4d16-b085-5dac7dcf9320 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:14.850853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:14.851530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:14.851862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:14.852148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] PATH_INFO: `/v3/projects/5a113c2463de448fa70bdd2919c7c89d/users/a8bda87df87b45df99fb9cc1c8d94252/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:14.873170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a113c2463de448fa70bdd2919c7c89d, user_id=a8bda87df87b45df99fb9cc1c8d94252, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:14.875885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:15.000767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1115/2227] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:14 2022] PUT /identity/v3/projects/5a113c2463de448fa70bdd2919c7c89d/users/a8bda87df87b45df99fb9cc1c8d94252/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:15.012113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:15.012410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:15.012701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:15.046445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] MFA Rules not processed for user `a8bda87df87b45df99fb9cc1c8d94252`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:15.047707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:15.111968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1113/2228] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:15.116477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:15.198439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad1490-620f-410e-9ede-bd00020cae1b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:15.213335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:15.213972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:15.214329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:15.214681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:15.215442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:15.217652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:15.356170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1116/2229] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:15.360245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:15.372338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860c35c-d557-42e2-aa6b-771f1ab13559 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:15.445079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:15.445787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:15.446144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:15.446493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:15.447217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:15.449369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:15.664371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1114/2230] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:15 2022] POST /identity/v3/users => generated 294 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:15.669204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:15.679765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c03b07-04ed-47fd-8086-40c0e9836463 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:15.684993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:15.685665 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:15.686005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:15.686290 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:15.687220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:15.688808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:15.706825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1117/2231] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:15.708382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:15.721470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947f72-d997-4ea9-8d5f-449b530a88b1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:15.732314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:15.732314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:15.732314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:15.732314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] PATH_INFO: `/v3/projects/fd9b2ac1a1b6419981701404091db8a9/users/67be992d289743cf894824f9513612c7/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:15.757851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9b2ac1a1b6419981701404091db8a9, user_id=67be992d289743cf894824f9513612c7, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:15.762129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:15.933203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1115/2232] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:15 2022] PUT /identity/v3/projects/fd9b2ac1a1b6419981701404091db8a9/users/67be992d289743cf894824f9513612c7/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:15.934196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:15.951953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db335c-17dc-4662-997d-7aab83d772f5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:15.965367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:16.039205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:16.040033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:16.040331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:16.040652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:16.041943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:16.046514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980bb27-9f1f-4bf9-8383-6c2ae983c705 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:16.061896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1118/2233] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:16.063339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:16.074642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3af6d-8e91-434a-b249-a0285ca24c8a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:16.080263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:16.081031 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:16.081314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:16.081641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] PATH_INFO: `/v3/projects/fd9b2ac1a1b6419981701404091db8a9/users/67be992d289743cf894824f9513612c7/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:16.084682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9b2ac1a1b6419981701404091db8a9, user_id=67be992d289743cf894824f9513612c7, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:16.087414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:16.099906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1116/2234] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:16 2022] PUT /identity/v3/projects/fd9b2ac1a1b6419981701404091db8a9/users/67be992d289743cf894824f9513612c7/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:16.115823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:16.116037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:16.116264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:16.145733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] MFA Rules not processed for user `67be992d289743cf894824f9513612c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:16.146973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:16.226043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1119/2235] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:16.236728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:16.238550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:16.301856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24676540-4472-4262-b0d2-e6bd25829f49 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:16.314395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:16.315108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:16.315397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:16.315708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:16.432658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:16.434166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead0b29-10c1-4c8b-aa1a-75e6a4250a54 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:16.458894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1117/2236] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:17.306028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:18.205776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:18.207448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:18.293751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b672c79-6286-4685-a787-a6ac89e48849 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:18.298879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:18.299554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:18.300131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:18.300131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] PATH_INFO: `/v3/users/35b1f88d727a4ab191c6602c2c3d06c9` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:18.301266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b1f88d727a4ab191c6602c2c3d06c9)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:18.303420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:18.737715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b1f88d727a4ab191c6602c2c3d06c9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:18.827294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b1f88d727a4ab191c6602c2c3d06c9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:18.828012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b1f88d727a4ab191c6602c2c3d06c9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:18.828766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b1f88d727a4ab191c6602c2c3d06c9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:18.847022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b1f88d727a4ab191c6602c2c3d06c9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:18.873544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1120/2237] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:18 2022] DELETE /identity/v3/users/35b1f88d727a4ab191c6602c2c3d06c9 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:18.879323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f42d4-799c-4303-9334-1faba9ba6d8a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:18.880261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f42d4-799c-4303-9334-1faba9ba6d8a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:18.880686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f42d4-799c-4303-9334-1faba9ba6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:18.952941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1d2f42d4-799c-4303-9334-1faba9ba6d8a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:18.959499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1d2f42d4-799c-4303-9334-1faba9ba6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:19.073033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:19.078365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1118/2238] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:19.102269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:19.103758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:19.212867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679a27-f660-4c9e-b07a-9420120de0ed admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:19.216976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:19.217742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:19.218103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:19.218451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:19.237392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:19.239073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:19.251277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1121/2239] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:19.377200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:19.377868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:19.378792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:19.415744 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] MFA Rules not processed for user `cfbdd49e2fa64b428d957d9dcf1d6846`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:19.416976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:19.487051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1119/2240] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:19.501992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:19.503536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:19.606596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a6b9-fe82-4717-875e-ccd6696b6a7c service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:19.612659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:19.613278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:19.613655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:19.613940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:19.624780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:19.626514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:19.638086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1122/2241] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:19.902479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:19.904319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:19.921401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ae417-2ecb-4f5e-bf18-672e77cdaaa2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:19.935628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:19.936349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:19.938047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:19.938335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] PATH_INFO: `/v3/projects/25f601a4b4184d858fa699a0ab0bf6b4` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:19.939992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f601a4b4184d858fa699a0ab0bf6b4)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:19.942330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:20.103045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f601a4b4184d858fa699a0ab0bf6b4', 'request_id': 'req-18204f64-cdf0-4336-881b-ab6bbb21e6ff'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.109870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f601a4b4184d858fa699a0ab0bf6b4', 'request_id': 'req-18204f64-cdf0-4336-881b-ab6bbb21e6ff'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.261177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f601a4b4184d858fa699a0ab0bf6b4', 'request_id': 'req-18204f64-cdf0-4336-881b-ab6bbb21e6ff'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.272866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] The token cache is being invalidate because project 25f601a4b4184d858fa699a0ab0bf6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:20.273171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.273838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.275531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1120/2242] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:19 2022] DELETE /identity/v3/projects/25f601a4b4184d858fa699a0ab0bf6b4 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:20.279792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:20.291155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6e59-c218-4890-a297-9c3b2676d575 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:20.353643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:20.354277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:20.354559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:20.354833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] PATH_INFO: `/v3/users/879dab227b9c4e5a84cc2ee920476527` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:20.356007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=879dab227b9c4e5a84cc2ee920476527)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:20.359135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:20.708088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879dab227b9c4e5a84cc2ee920476527'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.808556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879dab227b9c4e5a84cc2ee920476527'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.809360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879dab227b9c4e5a84cc2ee920476527'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.810319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879dab227b9c4e5a84cc2ee920476527'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.825803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879dab227b9c4e5a84cc2ee920476527'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:20.840818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1123/2243] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:20 2022] DELETE /identity/v3/users/879dab227b9c4e5a84cc2ee920476527 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:21.187284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:21.726520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:21.728528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:21.744760 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18204f64-cdf0-4336-881b-ab6bbb21e6ff admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:21.825378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:21.825948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:21.826232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:21.826746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] PATH_INFO: `/v3/projects/150c88f4277f41bb87d459d6d7c2d3d5` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:21.828108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=150c88f4277f41bb87d459d6d7c2d3d5)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:21.830262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:22.073335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150c88f4277f41bb87d459d6d7c2d3d5', 'request_id': 'req-1586c757-76aa-42b2-a920-e18682a6e853'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.091836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150c88f4277f41bb87d459d6d7c2d3d5', 'request_id': 'req-1586c757-76aa-42b2-a920-e18682a6e853'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.262424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150c88f4277f41bb87d459d6d7c2d3d5', 'request_id': 'req-1586c757-76aa-42b2-a920-e18682a6e853'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.289893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] The token cache is being invalidate because project 150c88f4277f41bb87d459d6d7c2d3d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:22.291208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.292065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.294002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1121/2244] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:21 2022] DELETE /identity/v3/projects/150c88f4277f41bb87d459d6d7c2d3d5 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:22.298781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:22.300770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:22.330259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193aecb4-5613-4cc9-94d5-501d5f01dc59 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:22.461437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:22.462085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:22.462369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:22.462648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] PATH_INFO: `/v3/users/9408eb0a36234f6a90eb16dec190059d` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:22.465067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9408eb0a36234f6a90eb16dec190059d)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:22.468380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:22.821532 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9408eb0a36234f6a90eb16dec190059d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.872738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:22.881002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:22.937042 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9408eb0a36234f6a90eb16dec190059d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.939227 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9408eb0a36234f6a90eb16dec190059d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.940640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9408eb0a36234f6a90eb16dec190059d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.983019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9408eb0a36234f6a90eb16dec190059d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:22.997426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c757-76aa-42b2-a920-e18682a6e853 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:23.005303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:23.005925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:23.006204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:23.006529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:23.017186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:23.021277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1124/2245] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:22 2022] DELETE /identity/v3/users/9408eb0a36234f6a90eb16dec190059d => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:23.151402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:23.151402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97b0fd-3ea6-40a0-9f30-a3eaa85f8f6c service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:23.158790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1122/2246] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:23.237560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:23.239193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:23.343367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d6d4b-d077-4f5d-903a-9d80c987573e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:23.357021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:23.357021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:23.357021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:23.357021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] PATH_INFO: `/v3/users/bb05adc30c434714bbb301b308096d40` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:23.357312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb05adc30c434714bbb301b308096d40)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:23.357312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:23.728597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb05adc30c434714bbb301b308096d40'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:23.809660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb05adc30c434714bbb301b308096d40'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:23.810369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb05adc30c434714bbb301b308096d40'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:23.810977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb05adc30c434714bbb301b308096d40'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:23.826377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb05adc30c434714bbb301b308096d40'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:23.840892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1125/2247] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:23 2022] DELETE /identity/v3/users/bb05adc30c434714bbb301b308096d40 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:23.845596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:23.845892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:23.846384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:23.873477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:23.874704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:23.938519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1123/2248] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:23.943530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:23.948359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:23.950168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:23.952400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:23.964766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ec77-0001-45ce-a476-50349a36d814 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:23.970048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:23.971197 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:23.971491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:23.971790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] PATH_INFO: `/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:23.973062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c237ec7a3c45b6b6d574fd428a5533)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:23.975608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:24.021436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612ed63-da0c-4fe1-ac2e-30acb5f12d66 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:24.025579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:24.026232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.026514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.026789 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.036810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:24.038307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12df38-26a8-4c9e-afde-b12dca04302d service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:24.048467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1124/2249] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:24.049707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798ebe-c010-4cb3-ab0d-ec4a861dee0c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.049990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798ebe-c010-4cb3-ab0d-ec4a861dee0c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.050265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798ebe-c010-4cb3-ab0d-ec4a861dee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.070509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6d798ebe-c010-4cb3-ab0d-ec4a861dee0c None None] MFA Rules not processed for user `cfbdd49e2fa64b428d957d9dcf1d6846`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:24.071754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6d798ebe-c010-4cb3-ab0d-ec4a861dee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:24.136218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1125/2250] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:24.188506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c237ec7a3c45b6b6d574fd428a5533', 'request_id': 'req-25889617-484e-4c34-9c06-7e2e345265a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:24.195408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c237ec7a3c45b6b6d574fd428a5533', 'request_id': 'req-25889617-484e-4c34-9c06-7e2e345265a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:24.301805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c237ec7a3c45b6b6d574fd428a5533', 'request_id': 'req-25889617-484e-4c34-9c06-7e2e345265a2'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:24.314819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] The token cache is being invalidate because project e7c237ec7a3c45b6b6d574fd428a5533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:24.315733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:24.316630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-25889617-484e-4c34-9c06-7e2e345265a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:24.323883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1126/2251] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:23 2022] DELETE /identity/v3/projects/e7c237ec7a3c45b6b6d574fd428a5533 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:24.327193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.327617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.328135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.355444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:24.356849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:24.459234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1126/2252] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:24.467800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.468297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.468405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.492747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:24.494148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:24.511257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1127/2253] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:24.512043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:24.522112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fabf4-cd7a-437b-bbfc-afe616438eac None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:24.527141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:24.527782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.528069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.528738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.529052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:24.529362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:24.530965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:24.547719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1127/2254] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:28:24.556760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:24.568853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce44fbc-f037-45db-b74d-fa444d17c014 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:24.573871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:24.574733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.575225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.575766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.576639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:24.578763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:24.886005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1128/2255] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:24 2022] POST /identity/v3/projects => generated 367 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:24.892273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:24.897609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:24.899778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:24.902989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244c317-2d99-4ebf-a1df-84fd477be41a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:24.988216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:24.989246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:24.989841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:24.990175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:24.990937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:24.993087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.017685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787b07-3538-4183-bbca-e55fa04559bd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.023298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.024145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.024429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.024711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] PATH_INFO: `/v3/projects/27b00afc58f04d35836eff41fee1268e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.025852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b00afc58f04d35836eff41fee1268e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.027810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.163341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1128/2256] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:24 2022] POST /identity/v3/users => generated 284 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:25.167982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:25.179062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3a2e-9d28-4fcd-84bb-fd6d0ea59fc4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.184596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.185435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.185699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.185784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.186434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.188083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.201039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1129/2257] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:25.211491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:25.223286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47886e9b-d68d-4cdd-b285-d92a5b871910 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.229366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.230365 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.230747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.231069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.231774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.233460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.251745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1130/2258] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:25.255817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:25.263324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b00afc58f04d35836eff41fee1268e', 'request_id': 'req-c410166b-355d-42fe-aea6-bfc2ec617672'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:25.266257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d15f-d461-42a9-9cb9-9f67aa4ffbce admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.274362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b00afc58f04d35836eff41fee1268e', 'request_id': 'req-c410166b-355d-42fe-aea6-bfc2ec617672'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:25.342646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.343564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.343907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.344312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] PATH_INFO: `/v3/projects/b5421348bbdb4991bc476448605d1453/users/1a4871498a304a28af2096de34f61479/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.358779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5421348bbdb4991bc476448605d1453, user_id=1a4871498a304a28af2096de34f61479, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.361462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a132-99a4-42b1-914c-66a19f0beb06 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.476079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b00afc58f04d35836eff41fee1268e', 'request_id': 'req-c410166b-355d-42fe-aea6-bfc2ec617672'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:25.477134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:25.488667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1131/2259] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:25 2022] PUT /identity/v3/projects/b5421348bbdb4991bc476448605d1453/users/1a4871498a304a28af2096de34f61479/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:25.494738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] The token cache is being invalidate because project 27b00afc58f04d35836eff41fee1268e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:25.495038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:25.495732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:25.497384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1129/2260] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:24 2022] DELETE /identity/v3/projects/27b00afc58f04d35836eff41fee1268e => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:25.502190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.502378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.502654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.506247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:25.508316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:25.521472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410166b-355d-42fe-aea6-bfc2ec617672 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.533240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] MFA Rules not processed for user `1a4871498a304a28af2096de34f61479`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:25.534470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:25.578603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.579239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.579529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.579964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] PATH_INFO: `/v3/users/380f330a876b4ba39f3974b5d5e70fbd` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.581090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] RBAC: Authorizing `identity:delete_user(user_id=380f330a876b4ba39f3974b5d5e70fbd)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.583032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.602632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1132/2261] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:25.615163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:25.617154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:25.684422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659006b-3294-4c8f-ba21-2928e0177827 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:25.688097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:25.688710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:25.688995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:25.689268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:25.698679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:25.700179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:25.708891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1133/2262] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:26.035787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '380f330a876b4ba39f3974b5d5e70fbd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:26.226957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '380f330a876b4ba39f3974b5d5e70fbd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:26.227682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '380f330a876b4ba39f3974b5d5e70fbd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:26.228431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '380f330a876b4ba39f3974b5d5e70fbd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:26.244447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '380f330a876b4ba39f3974b5d5e70fbd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:26.258105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1130/2263] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:25 2022] DELETE /identity/v3/users/380f330a876b4ba39f3974b5d5e70fbd => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:26.468202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:26.471328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:26.527191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446f097-b298-472c-8c57-a25bd84766c2 service glance] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:26.582189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:26.582371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:26.582591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:26.582869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:26.584728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:26.665852 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:26.667846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:26.681953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1134/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:26.915607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:26.919587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:27.150334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bd4be-941f-420d-b526-62fed7d1059c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:27.155399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:27.156073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:27.156451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:27.156734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] PATH_INFO: `/v3/projects/21978ec05424410893c691528f348c2a` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:27.157960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=21978ec05424410893c691528f348c2a)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:27.159923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:27.420101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21978ec05424410893c691528f348c2a', 'request_id': 'req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:27.427433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21978ec05424410893c691528f348c2a', 'request_id': 'req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:27.718591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21978ec05424410893c691528f348c2a', 'request_id': 'req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:27.731320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] The token cache is being invalidate because project 21978ec05424410893c691528f348c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:27.731589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:27.732311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:27.733946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1131/2265] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:26 2022] DELETE /identity/v3/projects/21978ec05424410893c691528f348c2a => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:27.739384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:27.750925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32d66-a28d-4e28-adfd-81a345c17dcf service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:27.811907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:27.812548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:27.812831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:27.813105 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] PATH_INFO: `/v3/users/7bf0a20134b24264ad2ad23ad4839db8` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:27.814298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf0a20134b24264ad2ad23ad4839db8)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:27.816286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:28.327071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf0a20134b24264ad2ad23ad4839db8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:28.358431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:28.792340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf0a20134b24264ad2ad23ad4839db8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:28.792975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf0a20134b24264ad2ad23ad4839db8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:28.803859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf0a20134b24264ad2ad23ad4839db8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:28.803859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf0a20134b24264ad2ad23ad4839db8'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:28.823102 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1135/2266] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:27 2022] DELETE /identity/v3/users/7bf0a20134b24264ad2ad23ad4839db8 => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:29.125976 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:29.128024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:29.148738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a0ec-2c05-4f7e-91d4-4e776df9bdde admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:29.212576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:29.213585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:29.213874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:29.214446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] PATH_INFO: `/v3/users/df2a2ca41653426a89aded6986e94247` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:29.215362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2a2ca41653426a89aded6986e94247)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:29.217384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:29.726500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:29.728464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:29.734466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2a2ca41653426a89aded6986e94247'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:29.745533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198daed5-aca2-440d-9d5d-8f7d3649e925 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:29.752350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:29.753148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:29.753661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:29.754425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] PATH_INFO: `/v3/projects/29866eae47fd4461861b2153917a6f45` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:29.755706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29866eae47fd4461861b2153917a6f45)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:29.758200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:30.058027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2a2ca41653426a89aded6986e94247'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.058849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2a2ca41653426a89aded6986e94247'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.059471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2a2ca41653426a89aded6986e94247'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.073590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e454034a-9150-45da-8e67-b5af0749b516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2a2ca41653426a89aded6986e94247'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.086913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1132/2267] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:29 2022] DELETE /identity/v3/users/df2a2ca41653426a89aded6986e94247 => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:30.091191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.091483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.091818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.113817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:30.115043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:30.127809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29866eae47fd4461861b2153917a6f45', 'request_id': 'req-0840011a-a914-4bef-9a08-2109912948b4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.134477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29866eae47fd4461861b2153917a6f45', 'request_id': 'req-0840011a-a914-4bef-9a08-2109912948b4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.178684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1133/2268] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:30.188836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:30.190251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:30.268066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7568895-f821-48f9-97be-ad9e0591833c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:30.272406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:30.273975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.273975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.274162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.277655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29866eae47fd4461861b2153917a6f45', 'request_id': 'req-0840011a-a914-4bef-9a08-2109912948b4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.287448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:30.289757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421d29b-538a-452f-85d2-f20514a3f34f service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:30.296023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] The token cache is being invalidate because project 29866eae47fd4461861b2153917a6f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:30.296330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.297121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0840011a-a914-4bef-9a08-2109912948b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:30.302665 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1136/2269] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:29 2022] DELETE /identity/v3/projects/29866eae47fd4461861b2153917a6f45 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:30.304718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.305473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1134/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:30.315011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.315011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.348434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:30.349759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:30.365427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1137/2271] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:30.366999 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.367375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.367733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.391120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:30.392373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:30.401250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1135/2272] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:30.405724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:30.416629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a695-bffe-4ea7-996c-ea1f0e868313 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:30.421623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:30.422242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.423415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.423415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.423415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:30.425219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:30.427327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:30.439631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1138/2273] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:28:30.444284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:30.453664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69024de1-d49e-4891-9fa2-ae446c252c57 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:30.458405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:30.459001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.459282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.459556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.460280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:30.462680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:30.794738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1136/2274] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:30 2022] POST /identity/v3/projects => generated 369 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:30.799047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:30.818862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe07bd-37e9-4fa9-802b-7225cd05cd4a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:30.924282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:30.924945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:30.925421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:30.925837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:30.926641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:30.928817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:30.991470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:30.994275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:31.010686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a574e-3b0a-4c4c-b4a0-d07a2b45c05e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:31.019782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:31.020504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.020791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.021068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] PATH_INFO: `/v3/projects/92494f2ed3104d41b5a84bdc2ddf3ebb` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.024654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92494f2ed3104d41b5a84bdc2ddf3ebb)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:31.026582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:31.170559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1139/2275] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:30 2022] POST /identity/v3/users => generated 285 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:31.174397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:31.185840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7975-c3e9-44ee-b283-f10cdf4f6ce9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:31.190690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:31.191292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.191576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.191894 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.192539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:31.194134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:31.205653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1140/2276] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:31.209711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:31.219502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203b6d-671b-42e4-9729-e0ff1eace18f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:31.224613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:31.224864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.225154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.225434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.226635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:31.228286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:31.241516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1141/2277] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:31.243132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:31.253161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632f224-39a8-4595-a222-7e55c0cb4158 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:31.258103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:31.258771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.259060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.259342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] PATH_INFO: `/v3/projects/2d548ac70c184bc1883d130d11ccecb5/users/7f6ad379f75a4caa94cc199830633e43/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.275220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d548ac70c184bc1883d130d11ccecb5, user_id=7f6ad379f75a4caa94cc199830633e43, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:31.277915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56708507-792f-4515-9133-11ec3a238e35 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:31.402958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1142/2278] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:31 2022] PUT /identity/v3/projects/2d548ac70c184bc1883d130d11ccecb5/users/7f6ad379f75a4caa94cc199830633e43/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:31.404750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.405107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.405515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.424939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92494f2ed3104d41b5a84bdc2ddf3ebb', 'request_id': 'req-a72db2c0-9d66-4919-a8cf-3e23dab038d6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:31.424939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92494f2ed3104d41b5a84bdc2ddf3ebb', 'request_id': 'req-a72db2c0-9d66-4919-a8cf-3e23dab038d6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:31.436985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] MFA Rules not processed for user `7f6ad379f75a4caa94cc199830633e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:31.438214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:31.502932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1143/2279] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:31.514739 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:31.516087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:31.575203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59de4e-d48e-4c5d-8385-ea380ded240e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:31.578863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89535015-3796-490f-be97-cf01089fe744 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:31.579474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535015-3796-490f-be97-cf01089fe744 service glance] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:31.579790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535015-3796-490f-be97-cf01089fe744 service glance] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:31.580068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535015-3796-490f-be97-cf01089fe744 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:31.590443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535015-3796-490f-be97-cf01089fe744 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:31.591987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535015-3796-490f-be97-cf01089fe744 service glance] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:31.613848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1144/2280] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:32.081474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92494f2ed3104d41b5a84bdc2ddf3ebb', 'request_id': 'req-a72db2c0-9d66-4919-a8cf-3e23dab038d6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:32.095442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] The token cache is being invalidate because project 92494f2ed3104d41b5a84bdc2ddf3ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:32.097404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:32.099446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a72db2c0-9d66-4919-a8cf-3e23dab038d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:32.102165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1137/2281] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:30 2022] DELETE /identity/v3/projects/92494f2ed3104d41b5a84bdc2ddf3ebb => generated 0 bytes in 1111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:32.108570 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:32.109071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:32.109354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:32.130474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:32.132460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:32.206602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1145/2282] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:32.219124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:32.219666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:32.220029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:32.243541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:32.244823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:32.253878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1138/2283] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:32.258386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:32.271329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ee6cc-6f98-49b0-a194-8204bfebf292 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:32.276721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:32.277336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:32.277648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:32.277924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:32.278580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:32.278885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:32.280526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:32.295875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1146/2284] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:32.299518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:32.310018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b119a-07ad-41cf-b3b7-b6dcf770ad32 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:32.315128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:32.315775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:32.316060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:32.316332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:32.316968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:32.318900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:33.212198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1139/2285] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:32 2022] POST /identity/v3/projects => generated 383 bytes in 910 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:33.214028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:33.225051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4031e69-57dd-438b-9f26-265ef5ff15cf admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:33.322202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:33.322922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:33.323280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:33.323618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:33.324538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:33.326751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:33.548097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1147/2286] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:33 2022] POST /identity/v3/users => generated 292 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:33.552102 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:33.563279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a85bf-2a0e-43e5-8cb7-55f7b60bc4a8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:33.567924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:33.568526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:33.568810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:33.569092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:33.569737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:33.571335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:33.581707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1140/2287] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:33.585801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:33.596333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b89c0c-a67e-4be1-b6b7-0163d0e42c57 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:33.601756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:33.602440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:33.602786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:33.603119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:33.603885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:33.613435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f673-135c-4488-9a49-59bf4744e34a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:33.626535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1148/2288] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:33.630559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:33.642533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dafe7-78c8-4078-af9b-4f24de3df9c4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:33.647639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:33.648388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:33.648671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:33.648941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] PATH_INFO: `/v3/projects/4fd3f2da26af4d3db4c6db4ceb851558/users/fd8d3fe4be354995b1698b9783ce6c65/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:33.665882 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd3f2da26af4d3db4c6db4ceb851558, user_id=fd8d3fe4be354995b1698b9783ce6c65, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:33.668581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:33.827130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1141/2289] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:33 2022] PUT /identity/v3/projects/4fd3f2da26af4d3db4c6db4ceb851558/users/fd8d3fe4be354995b1698b9783ce6c65/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:33.838369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:33.839298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:33.839784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:33.869902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:33.871685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:33.992498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1149/2290] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:34.004067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:34.015855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:34.107939 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1618-1c97-45d1-b0dc-9adbe757e7f3 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:34.111774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:34.112686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:34.112967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:34.113238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:34.124321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:34.125957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582ae6-6a36-4619-9d2a-5d61ba6a2522 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:34.137341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1142/2291] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:34.892951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:35.073386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:35.119240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:35.121118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:35.134956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3f9b-83eb-4ecf-b5a7-60b09d5ce299 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:35.141324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:35.141977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:35.142283 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:35.142565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] PATH_INFO: `/v3/users/7a76979b6ce24ebc9c0bea8d3e0a150b` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:35.143764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a76979b6ce24ebc9c0bea8d3e0a150b)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:35.145684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:35.678500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a76979b6ce24ebc9c0bea8d3e0a150b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:35.901897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a76979b6ce24ebc9c0bea8d3e0a150b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:35.907366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a76979b6ce24ebc9c0bea8d3e0a150b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:35.908712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a76979b6ce24ebc9c0bea8d3e0a150b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:35.920454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a76979b6ce24ebc9c0bea8d3e0a150b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:35.943417 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1150/2292] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:35 2022] DELETE /identity/v3/users/7a76979b6ce24ebc9c0bea8d3e0a150b => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:35.948769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:35.949124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:35.949468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:35.969327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:35.970790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:36.033817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1143/2293] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:36.043932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:36.045424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:36.123215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f58be-fe58-4d6b-a04d-97671dbb74f2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:36.132568 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:36.134218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:36.134551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:36.134836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:36.145491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:36.147127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:36.158059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1151/2294] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:36.880848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:36.885656 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:36.905050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a5a81-e379-44f5-8be6-596deb60f6d3 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:36.918765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:36.919470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:36.919792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:36.920246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] PATH_INFO: `/v3/projects/0f40505192a74e1793f7562e6b8e7fc7` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:36.924499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f40505192a74e1793f7562e6b8e7fc7)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:36.926502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:37.143710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f40505192a74e1793f7562e6b8e7fc7', 'request_id': 'req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.161232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f40505192a74e1793f7562e6b8e7fc7', 'request_id': 'req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.203520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:37.206326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:37.223355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe046-9dd9-4e3a-a846-419caed804b7 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:37.279494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f40505192a74e1793f7562e6b8e7fc7', 'request_id': 'req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.296343 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] The token cache is being invalidate because project 0f40505192a74e1793f7562e6b8e7fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:37.296868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.297722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.303565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1144/2295] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:36 2022] DELETE /identity/v3/projects/0f40505192a74e1793f7562e6b8e7fc7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:37.305273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:37.307031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:37.340003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:37.345095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:37.345095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:37.345095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] PATH_INFO: `/v3/users/aa31273cf7a04f4287b5545c89176bfc` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:37.345095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa31273cf7a04f4287b5545c89176bfc)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:37.345701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:37.407792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2d2ef-7184-459c-ba1b-77e27fc04e4e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:37.413967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:37.414762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:37.419643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:37.419643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] PATH_INFO: `/v3/users/a8bda87df87b45df99fb9cc1c8d94252` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:37.419643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bda87df87b45df99fb9cc1c8d94252)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:37.419643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:37.769442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa31273cf7a04f4287b5545c89176bfc'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.785056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bda87df87b45df99fb9cc1c8d94252'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.906773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa31273cf7a04f4287b5545c89176bfc'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.907239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa31273cf7a04f4287b5545c89176bfc'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.907981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa31273cf7a04f4287b5545c89176bfc'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.937454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1f781610-5b43-4666-9ec6-9018516f6775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa31273cf7a04f4287b5545c89176bfc'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:37.963511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1152/2296] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:37 2022] DELETE /identity/v3/users/aa31273cf7a04f4287b5545c89176bfc => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:37.968456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:37.968791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:37.969014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:37.998988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:38.000310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:38.050039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bda87df87b45df99fb9cc1c8d94252'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:38.050745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bda87df87b45df99fb9cc1c8d94252'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:38.051368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bda87df87b45df99fb9cc1c8d94252'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:38.078475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bda87df87b45df99fb9cc1c8d94252'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:38.111703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1145/2297] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:37 2022] DELETE /identity/v3/users/a8bda87df87b45df99fb9cc1c8d94252 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:38.162621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1153/2298] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:38.181746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:38.184012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:38.325181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6a550-9aa0-4905-ab05-51cc7ca3a302 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:38.334022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:38.334963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:38.335252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:38.336233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:38.447055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:38.448546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:38.463298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1146/2299] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:39.330641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:39.333756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:39.357493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e58eb-5d02-42ff-9e57-1872f37ac145 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:39.364989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:39.366215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:39.372794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:39.372794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] PATH_INFO: `/v3/projects/5a113c2463de448fa70bdd2919c7c89d` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:39.372794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a113c2463de448fa70bdd2919c7c89d)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:39.373541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:39.470481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:39.472518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:39.511798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fad06-a17d-4c70-8faa-38a3198e47d8 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:39.518347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:39.519159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:39.519741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:39.520116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] PATH_INFO: `/v3/projects/7f0f6d5fc07746908fea709b8c261521` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:39.521403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0f6d5fc07746908fea709b8c261521)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:39.524005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:39.654428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a113c2463de448fa70bdd2919c7c89d', 'request_id': 'req-65834456-81bd-4026-97b3-ff8cb3937aa0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.665436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a113c2463de448fa70bdd2919c7c89d', 'request_id': 'req-65834456-81bd-4026-97b3-ff8cb3937aa0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.739456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0f6d5fc07746908fea709b8c261521', 'request_id': 'req-10caa856-1400-410f-a798-7c71f5ff7600'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.751551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0f6d5fc07746908fea709b8c261521', 'request_id': 'req-10caa856-1400-410f-a798-7c71f5ff7600'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.839531 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a113c2463de448fa70bdd2919c7c89d', 'request_id': 'req-65834456-81bd-4026-97b3-ff8cb3937aa0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.860247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] The token cache is being invalidate because project 5a113c2463de448fa70bdd2919c7c89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:39.860734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.861817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.866596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1154/2300] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:39 2022] DELETE /identity/v3/projects/5a113c2463de448fa70bdd2919c7c89d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:39.875871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:39.877679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:39.908678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0f6d5fc07746908fea709b8c261521', 'request_id': 'req-10caa856-1400-410f-a798-7c71f5ff7600'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.937925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] The token cache is being invalidate because project 7f0f6d5fc07746908fea709b8c261521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:39.939206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.940559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-10caa856-1400-410f-a798-7c71f5ff7600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:39.942817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1147/2301] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:39 2022] DELETE /identity/v3/projects/7f0f6d5fc07746908fea709b8c261521 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:39.950179 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd508-53a5-47fc-903a-caac4404cd30 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:39.950787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd508-53a5-47fc-903a-caac4404cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:39.951251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd508-53a5-47fc-903a-caac4404cd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:39.979965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6ccdd508-53a5-47fc-903a-caac4404cd30 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:39.981557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6ccdd508-53a5-47fc-903a-caac4404cd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:40.008777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65834456-81bd-4026-97b3-ff8cb3937aa0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.087328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1148/2302] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:40.087550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.087550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.087827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.088063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] PATH_INFO: `/v3/users/67be992d289743cf894824f9513612c7` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.089242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67be992d289743cf894824f9513612c7)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.089662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.090033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.090363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.091194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.111301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:40.112979 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:40.123433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1149/2303] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:40.128379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.139876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4b8b-49d8-4d13-b5e7-f5bc8a2274ed None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.145768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.146486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.146830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.147116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.148048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:40.148517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.151053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.165794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1150/2304] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:40.170963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.189601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd82f-08a2-4199-b306-f9a431b76b15 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.196105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.196861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.197416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.197793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.198578 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.200734 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.319059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:40.377557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1151/2305] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:40 2022] POST /identity/v3/projects => generated 361 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:40.381653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.407471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67be992d289743cf894824f9513612c7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:40.461892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7703bc-e8de-49c1-8405-2650280037de admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.468223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.468560 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.468851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.469114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.469754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.471691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.615549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67be992d289743cf894824f9513612c7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:40.616536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67be992d289743cf894824f9513612c7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:40.617969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67be992d289743cf894824f9513612c7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:40.628891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67be992d289743cf894824f9513612c7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:40.643502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1152/2306] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:40 2022] POST /identity/v3/users => generated 281 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:28:40.645742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.649361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:40.666628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e8b4-199f-44f0-aff2-fe36908893e8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.670090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1155/2307] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:39 2022] DELETE /identity/v3/users/67be992d289743cf894824f9513612c7 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:40.676766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.677758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.678266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.678705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.680068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.682326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.716328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1153/2308] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:40.721683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.864302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9b92-1655-49f8-a96a-328b9cf755f3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.879975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.880420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.880842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.881191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:40.881859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:40.887895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6ef9-4e71-4306-91a6-62d0f8dda2b2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:40.918524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:40.923701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1156/2309] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:40.949864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc472c-12bf-4652-870a-30071070af98 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:40.955139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:40.955853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:40.956137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:40.956416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] PATH_INFO: `/v3/projects/d7ffad0adc9b46dc872e4049fb90f43c/users/064633cd65dd4d22930692f8cbf03aeb/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:41.048834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ffad0adc9b46dc872e4049fb90f43c, user_id=064633cd65dd4d22930692f8cbf03aeb, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:41.056452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:41.212857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1154/2310] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:40 2022] PUT /identity/v3/projects/d7ffad0adc9b46dc872e4049fb90f43c/users/064633cd65dd4d22930692f8cbf03aeb/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:41.218303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431871-a644-4bed-8358-adbadf03231d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:41.218791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431871-a644-4bed-8358-adbadf03231d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:41.219140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431871-a644-4bed-8358-adbadf03231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:41.263145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3f431871-a644-4bed-8358-adbadf03231d None None] MFA Rules not processed for user `064633cd65dd4d22930692f8cbf03aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:41.265176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f431871-a644-4bed-8358-adbadf03231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:41.367729 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1157/2311] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:41.422278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:41.424775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:41.464850 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431871-a644-4bed-8358-adbadf03231d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:41.466978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f431871-a644-4bed-8358-adbadf03231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:41.572154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431871-a644-4bed-8358-adbadf03231d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:41.575362 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:41.576267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:41.578795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:41.579116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:41.592049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33d04-b287-4d00-9b5a-f56bf9da31e9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:41.598131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:41.601877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:41.601877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:41.601877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] PATH_INFO: `/v3/projects/fd9b2ac1a1b6419981701404091db8a9` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:41.601877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9b2ac1a1b6419981701404091db8a9)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:41.604555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:41.628256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:41.635265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:41.667415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1158/2312] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:41.683482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:41.704475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3f3a8-d919-4a1c-a920-bfb27c16f7ea service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:41.712596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:41.713490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:41.713982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:41.714925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] PATH_INFO: `/v3/users/064633cd65dd4d22930692f8cbf03aeb` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:41.716629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=064633cd65dd4d22930692f8cbf03aeb)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:41.719471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:41.902335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9b2ac1a1b6419981701404091db8a9', 'request_id': 'req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:41.917715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9b2ac1a1b6419981701404091db8a9', 'request_id': 'req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.199000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9b2ac1a1b6419981701404091db8a9', 'request_id': 'req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.215625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] The token cache is being invalidate because project fd9b2ac1a1b6419981701404091db8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:42.216510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064633cd65dd4d22930692f8cbf03aeb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.216632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.217223 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3e115e96-addc-4f1b-aeea-740e7a4a1aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.224525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1155/2313] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:41 2022] DELETE /identity/v3/projects/fd9b2ac1a1b6419981701404091db8a9 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:42.230815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40624884-5044-441f-af3b-9e0412d46341 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.231105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40624884-5044-441f-af3b-9e0412d46341 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.231386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40624884-5044-441f-af3b-9e0412d46341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.290218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-40624884-5044-441f-af3b-9e0412d46341 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:42.291745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-40624884-5044-441f-af3b-9e0412d46341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:42.341467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064633cd65dd4d22930692f8cbf03aeb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.342114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064633cd65dd4d22930692f8cbf03aeb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.342735 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064633cd65dd4d22930692f8cbf03aeb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.355455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ce057d79-50da-4334-a71a-fbb6ae15ac9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064633cd65dd4d22930692f8cbf03aeb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:42.391116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1159/2314] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:41 2022] DELETE /identity/v3/users/064633cd65dd4d22930692f8cbf03aeb => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:42.394234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.394543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.395057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.431475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1156/2315] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:42.437433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:42.440794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:42.442720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.444416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.445410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.497168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:42.499204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:42.577606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1160/2316] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:42.588450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:42.590195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:42.612325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1157/2317] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:42.613274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:42.630195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5a68b-d721-478c-828a-3bb4e8084e34 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:42.640114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:42.641563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.641848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.642126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.643561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:42.643899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:42.646501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:42.702089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1158/2318] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:28:42.706846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:42.751322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec247-f918-4838-a021-11bc8385ab01 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:42.753396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874bde4-27bb-49cd-916c-4daf41565777 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:42.756893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:42.758273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.758748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.759641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.774958 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:42.775916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:42.777190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:42.780481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:42.781009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:42.781120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:42.781811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:42.783763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:42.796359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1161/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:42.879398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:43.031828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1159/2320] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:42 2022] POST /identity/v3/projects => generated 399 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:43.059550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.075939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1594-1106-475c-acf5-231843580cad service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:43.201244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:43.204055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:43.205337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:43.205337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:43.206547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:43.209188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:43.330164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1162/2321] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:43 2022] POST /identity/v3/users => generated 300 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:43.334326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.361681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a9dd3-0d14-44ad-be27-10c7e6f0fe23 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:43.374419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:43.375249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:43.383745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:43.383745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:43.383745 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:43.393894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:43.408925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1160/2322] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:43.413412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.438059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5597-dd7d-4d8a-b672-4383252c3a2c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:43.444477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:43.445308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:43.445719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:43.446010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:43.446676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:43.450022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9792fbd-75ff-4330-aa73-92ffbb624a68 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:43.497479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1163/2323] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:43.502614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.519444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c468e81-7761-4df1-b87d-08c51f4839a9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:43.525919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:43.526837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:43.527272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:43.527755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] PATH_INFO: `/v3/projects/b4e88719bf024b63b8da001c0e9497bf/users/98f10b1a62e34eccb026f6f0b4a6d5e6/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:43.552272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e88719bf024b63b8da001c0e9497bf, user_id=98f10b1a62e34eccb026f6f0b4a6d5e6, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:43.557350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:43.697340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1161/2324] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:43 2022] PUT /identity/v3/projects/b4e88719bf024b63b8da001c0e9497bf/users/98f10b1a62e34eccb026f6f0b4a6d5e6/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:43.709213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:43.709869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:43.709869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:43.747911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] MFA Rules not processed for user `98f10b1a62e34eccb026f6f0b4a6d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:43.750216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:43.813047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.815084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:43.973960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1164/2325] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:43.986658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:43.990980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:44.020837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63148904-7d8e-4d76-94d7-206018240b96 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:44.037069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:44.061715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.063515 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.064958 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] PATH_INFO: `/v3/projects/d7ffad0adc9b46dc872e4049fb90f43c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.066905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ffad0adc9b46dc872e4049fb90f43c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:44.069546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:44.101181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00d7e8-3456-4557-b252-435f01d61769 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:44.105806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:44.106442 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.106733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.107013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.118591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:44.120302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd0eb1-f5ff-4936-91fd-1e19e5ed7252 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:44.160280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1165/2326] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:44.275117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ffad0adc9b46dc872e4049fb90f43c', 'request_id': 'req-b33e47f6-9188-4136-ac25-dde811324cb0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:44.291839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ffad0adc9b46dc872e4049fb90f43c', 'request_id': 'req-b33e47f6-9188-4136-ac25-dde811324cb0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:44.528392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ffad0adc9b46dc872e4049fb90f43c', 'request_id': 'req-b33e47f6-9188-4136-ac25-dde811324cb0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:44.563782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] The token cache is being invalidate because project d7ffad0adc9b46dc872e4049fb90f43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:44.563782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:44.563782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b33e47f6-9188-4136-ac25-dde811324cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:44.564945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1162/2327] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:43 2022] DELETE /identity/v3/projects/d7ffad0adc9b46dc872e4049fb90f43c => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:44.571355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.571355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.571355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.603537 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:44.605020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:44.752683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1166/2328] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:44.761043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.761776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.761776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.791700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:44.792971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:44.813463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1163/2329] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:44.817354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:44.830839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26b1ea-f39e-4511-847b-e55b9c09579b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:44.838802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:44.839692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.840331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.841013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.841938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:28:44.842421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:44.844794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7428e-0b10-4527-8dc4-ac4789275d5a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:44.869839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1167/2330] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:28:44.893044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:44.904616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a7574-3f29-40ee-ac85-b53dfa9d9761 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:44.915241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:44.916237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:44.918202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:44.918558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:44.919281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:44.926575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.050989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.051492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.051943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.107418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] MFA Rules not processed for user `fd8d3fe4be354995b1698b9783ce6c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:45.108972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:45.198097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1164/2331] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:44 2022] POST /identity/v3/projects => generated 373 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:45.202536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.252959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1168/2332] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:28:45.256626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.258831 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:45.278303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aca64a-7bdc-4231-9b44-3c81bcbdc6ab None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.332370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7acad6-a146-43e9-9bca-c82ca63f1550 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.344449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.346501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.347356 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.347638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.349625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.366059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.417201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.417857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.418137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.418558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.419202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.421673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.642576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1169/2333] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:28:45 2022] POST /identity/v3/projects => generated 385 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:45.650112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.658122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1165/2334] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:45 2022] POST /identity/v3/users => generated 287 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:45.663672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.688812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1d67-d065-4d09-bb99-02d92c69e857 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.746489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3c6-e73e-488c-903c-588692675a7d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.754814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.755742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.756001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.756280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.756914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.759451 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.858202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.859128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.859425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.859746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.860613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.862261 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.886003 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1166/2335] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:45.890354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.917375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85be3-ee54-419b-856f-2f170be330ba admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.924522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1170/2336] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:28:45 2022] POST /identity/v3/users => generated 299 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:28:45.925135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.925783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.926076 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.926359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.927026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.929021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.930115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.955753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1167/2337] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:45.964199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:45.965130 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e743b-771e-4b61-af89-4333ecd47386 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.972152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.972820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.974384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.974672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.975503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:45.978425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:45.984036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb815d-4a13-4326-91f3-87b4abfb2ce2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:45.995774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1171/2338] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:28:45.995947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:45.996644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:45.997141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:45.997975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] PATH_INFO: `/v3/projects/cf90b9345ae84e26b58e9238da0dac2e/users/7de2f8a8c6644220b9e1ed37a48b7d9a/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:45.998244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:46.015320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7eeb-fdc7-4d5f-8564-fd2ca8be3e0a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:46.021056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:46.021872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:46.022256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:46.022635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] PATH_INFO: `/v3/projects/52ab88f388b84b5488a950739c0710d0/users/8639718ca51d40d7850cb4212d69c214/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:46.040459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf90b9345ae84e26b58e9238da0dac2e, user_id=7de2f8a8c6644220b9e1ed37a48b7d9a, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:46.042000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ab88f388b84b5488a950739c0710d0, user_id=8639718ca51d40d7850cb4212d69c214, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:46.043638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91780f-b98e-40ee-9b9a-31ba15e66c32 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:46.046459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:46.171966 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1172/2339] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:45 2022] PUT /identity/v3/projects/52ab88f388b84b5488a950739c0710d0/users/8639718ca51d40d7850cb4212d69c214/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:46.178145 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:46.217236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1168/2340] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:45 2022] PUT /identity/v3/projects/cf90b9345ae84e26b58e9238da0dac2e/users/7de2f8a8c6644220b9e1ed37a48b7d9a/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:46.224825 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:46.225379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:46.225846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:46.248761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:46.250026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:46.256581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbdd7e-d47a-4e38-a2c3-4425be05bcef admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:46.322619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:46.323363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:46.323741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:46.324172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:46.324890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:46.327306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1169/2341] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:46.327450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:46.344962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:46.346561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:46.347114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1173/2342] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:28:46.352431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:46.364803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756db1b-7613-4f88-8a9f-a73d4a78deb4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:46.370217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:46.370900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:46.371336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:46.371609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] PATH_INFO: `/v3/projects/52ab88f388b84b5488a950739c0710d0/users/8639718ca51d40d7850cb4212d69c214/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:46.374330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ab88f388b84b5488a950739c0710d0, user_id=8639718ca51d40d7850cb4212d69c214, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:46.376687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106838-4ea2-419d-9a85-4f262c468071 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:46.394239 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1174/2343] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:28:46 2022] PUT /identity/v3/projects/52ab88f388b84b5488a950739c0710d0/users/8639718ca51d40d7850cb4212d69c214/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:46.426838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd2926-2436-4de4-9013-64443dc5e10a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:46.482374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:46.483022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:46.483306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:46.483746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:46.547287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:46.550127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:46.561195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1170/2344] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:28:46.621101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:48.886282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:49.723057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a01adc-96c0-4285-907d-cdb34ce49c10 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:49.723586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a01adc-96c0-4285-907d-cdb34ce49c10 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:49.723586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a01adc-96c0-4285-907d-cdb34ce49c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:49.756532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c8a01adc-96c0-4285-907d-cdb34ce49c10 None None] MFA Rules not processed for user `7de2f8a8c6644220b9e1ed37a48b7d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:49.757885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c8a01adc-96c0-4285-907d-cdb34ce49c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:49.831500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1175/2345] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:49.849196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:49.851766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:49.936040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc336-1a5d-4568-9ab5-6fd557be3415 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:49.940244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:49.940864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:49.941151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:49.941431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:49.950435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:49.952110 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2d488-bddb-490e-9984-448510a2e2ca service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:49.962508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1171/2346] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:50.371952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:53.249788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:53.320218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:53.320900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:53.321004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:53.360911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] MFA Rules not processed for user `7de2f8a8c6644220b9e1ed37a48b7d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:53.363518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:53.379915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1176/2347] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:53.940346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa2a14-4a02-4711-a101-d3b111c7a41e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:53.940635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa2a14-4a02-4711-a101-d3b111c7a41e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:53.940935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa2a14-4a02-4711-a101-d3b111c7a41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:53.968300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-46aa2a14-4a02-4711-a101-d3b111c7a41e None None] MFA Rules not processed for user `0e37c8ca70ad4b6e8430a71d0f5e564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:53.969554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-46aa2a14-4a02-4711-a101-d3b111c7a41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:54.076691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1172/2348] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:54.454898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:56.182779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:56.329499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:56.330926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:56.364553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf45e9-4443-4a72-818f-1b9b0eda8074 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:56.387596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:56.388651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:56.389147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:56.390561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] PATH_INFO: `/v3/users/98f10b1a62e34eccb026f6f0b4a6d5e6` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:56.396215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f10b1a62e34eccb026f6f0b4a6d5e6)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:56.398795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:56.543196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:56.543823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:56.544390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:56.619524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] MFA Rules not processed for user `8639718ca51d40d7850cb4212d69c214`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:56.621004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:56.716440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f10b1a62e34eccb026f6f0b4a6d5e6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:56.745105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1173/2349] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:56.762965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:56.774092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:56.859033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f10b1a62e34eccb026f6f0b4a6d5e6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:56.860219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f10b1a62e34eccb026f6f0b4a6d5e6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:56.861358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f10b1a62e34eccb026f6f0b4a6d5e6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:56.876236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-d513fea6-e2b9-4abd-b682-7a513566036f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f10b1a62e34eccb026f6f0b4a6d5e6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:56.882349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344caee-2b6e-435b-832a-a8caba53cbe8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:56.886478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:56.887372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:56.887881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:56.887980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:56.907059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:56.949457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1177/2350] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:28:56 2022] DELETE /identity/v3/users/98f10b1a62e34eccb026f6f0b4a6d5e6 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:28:56.951372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c995dc-ae5d-49e6-bead-579e942187d3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:56.951761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c995dc-ae5d-49e6-bead-579e942187d3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:56.952347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c995dc-ae5d-49e6-bead-579e942187d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:56.980810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-51c995dc-ae5d-49e6-bead-579e942187d3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:56.982949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-51c995dc-ae5d-49e6-bead-579e942187d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:57.018638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:57.019690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:57.041008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1174/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:57.092603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1178/2352] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:57.105445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:57.106730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:57.109333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:57.125282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bffdee-52da-4228-90ed-f1d26f296a60 service glance] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:57.193370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:57.196696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:57.196696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:57.196696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:57.207744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:57.209079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:57.220952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1175/2353] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:28:57.546774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:57.547590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:57.547590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:57.579445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:28:57.582080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:57.753276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1179/2354] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:28:57.841131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:57.842701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:57.993831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128605bb-100b-46b5-80d4-accba26a4c5b service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:58.006015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:28:58.008818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:28:58.010691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:58.011548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:58.012026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:58.012620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:58.028526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:58.045094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031e58-cd83-4d64-acf2-dd9e951d3297 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:58.050043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392008f2-35f7-47bf-a002-5e7449287cc7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:28:58.067021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:28:58.067707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:28:58.068000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:28:58.068275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] PATH_INFO: `/v3/projects/b4e88719bf024b63b8da001c0e9497bf` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:28:58.070582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e88719bf024b63b8da001c0e9497bf)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:28:58.072944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:28:58.078220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1176/2355] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:28:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:28:58.271277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e88719bf024b63b8da001c0e9497bf', 'request_id': 'req-fc348e39-c615-4db2-bed7-d55df189b59b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:58.283954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e88719bf024b63b8da001c0e9497bf', 'request_id': 'req-fc348e39-c615-4db2-bed7-d55df189b59b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:58.430291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e88719bf024b63b8da001c0e9497bf', 'request_id': 'req-fc348e39-c615-4db2-bed7-d55df189b59b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:58.453083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] The token cache is being invalidate because project b4e88719bf024b63b8da001c0e9497bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:28:58.453868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:58.454864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fc348e39-c615-4db2-bed7-d55df189b59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:28:58.457044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1180/2356] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:28:58 2022] DELETE /identity/v3/projects/b4e88719bf024b63b8da001c0e9497bf => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:28:58.954622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:28:58.985915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:03.928950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:05.058484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:08.266621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074929cf-9726-4929-9720-ef59cebbdcae None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:08.267484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074929cf-9726-4929-9720-ef59cebbdcae None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:08.267630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074929cf-9726-4929-9720-ef59cebbdcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:08.297899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-074929cf-9726-4929-9720-ef59cebbdcae None None] MFA Rules not processed for user `1d7c40b5232e46119434aeeb8a6ad4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:08.299176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-074929cf-9726-4929-9720-ef59cebbdcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:08.428304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1177/2357] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:08.820749 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:09.420094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:10.799359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:11.169206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:12.503699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:13.046983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:13.047505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:13.047619 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:13.073220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] MFA Rules not processed for user `fd8d3fe4be354995b1698b9783ce6c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:13.074440 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:13.131692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1181/2358] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:13.515380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:13.567733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:15.036712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:15.463305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:16.448465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:17.259946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:18.372306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:18.373735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:18.374259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:18.399147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:18.400565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:18.501283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1178/2359] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:29:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:29:18.520330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:18.529073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:18.646953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8757-5f31-40c8-b354-203f7b6a224f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:18.663346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:18.664390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:18.664938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:18.665453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:18.683238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:18.685218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:18.732751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1182/2360] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:29:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:29:19.243389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:19.339994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:19.932215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:19.934288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:19.941131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:19.943228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:20.093600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4d71-410c-4514-ad11-0808ea013418 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:20.096757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0d8-2241-4c09-bbe9-c1af2adb95ba service swift] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:20.105749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:20.112536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:20.112710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:20.112710 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:20.112930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:20.113028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] PATH_INFO: `/v3/users/7f6ad379f75a4caa94cc199830633e43` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:20.113125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:20.113125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] PATH_INFO: `/v3/users/1a4871498a304a28af2096de34f61479` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:20.113281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6ad379f75a4caa94cc199830633e43)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:20.113281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:20.113872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4871498a304a28af2096de34f61479)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:20.120169 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:20.256086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:20.430770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4871498a304a28af2096de34f61479'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.441939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6ad379f75a4caa94cc199830633e43'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.505116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4871498a304a28af2096de34f61479'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.505980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4871498a304a28af2096de34f61479'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.507256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4871498a304a28af2096de34f61479'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.537186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-23de1714-356a-462e-b657-c172a15a43ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4871498a304a28af2096de34f61479'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.569426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1179/2361] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:29:19 2022] DELETE /identity/v3/users/1a4871498a304a28af2096de34f61479 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:29:20.578761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:20.578761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:20.578761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:20.585047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6ad379f75a4caa94cc199830633e43'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.587860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6ad379f75a4caa94cc199830633e43'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.588801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6ad379f75a4caa94cc199830633e43'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.603537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:20.606350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8ee99753-94b2-4d7b-b6bf-95a63b6f498b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6ad379f75a4caa94cc199830633e43'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:20.627620 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:20.664231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1183/2362] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:29:19 2022] DELETE /identity/v3/users/7f6ad379f75a4caa94cc199830633e43 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:29:20.669680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:20.670389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:20.673325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:20.712457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:20.715795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:20.833148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1180/2363] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:20.843863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:20.845417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:20.884860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1184/2364] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:20.894084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:20.901939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:21.013226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5af8-5574-45f3-87cd-cf8b16869676 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:21.079340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcb35b-1717-49d5-b817-5b100c3fd13f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:21.104666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:21.105589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:21.107592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:21.108073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:21.127821 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:21.130067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:21.137678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:21.139240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:21.144088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:21.144088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:21.144609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1185/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:29:21.155817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:21.157768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:21.174366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1181/2366] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:29:21.865959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:22.033021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:22.520602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:22.523825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:22.538084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413a4f-7bf2-4a10-a1bf-dfd4667b90f3 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:22.544685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:22.545520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:22.545807 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:22.546336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] PATH_INFO: `/v3/projects/b5421348bbdb4991bc476448605d1453` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:22.547842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5421348bbdb4991bc476448605d1453)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:22.550019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:22.671777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:22.681172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:22.699211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d8c5-fa0a-4e4e-9cda-9962f2a858a0 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:22.701344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5421348bbdb4991bc476448605d1453', 'request_id': 'req-da057736-853d-4def-a268-4f48ceec2f90'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:22.713357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5421348bbdb4991bc476448605d1453', 'request_id': 'req-da057736-853d-4def-a268-4f48ceec2f90'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:22.790959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5421348bbdb4991bc476448605d1453', 'request_id': 'req-da057736-853d-4def-a268-4f48ceec2f90'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:22.826542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] The token cache is being invalidate because project b5421348bbdb4991bc476448605d1453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:29:22.827147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:22.828394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da057736-853d-4def-a268-4f48ceec2f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:22.846412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1186/2367] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:29:22 2022] DELETE /identity/v3/projects/b5421348bbdb4991bc476448605d1453 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:29:22.908490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:22.909286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:22.909712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:22.910058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] PATH_INFO: `/v3/projects/2d548ac70c184bc1883d130d11ccecb5` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:22.911379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d548ac70c184bc1883d130d11ccecb5)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:22.913664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.129057 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d548ac70c184bc1883d130d11ccecb5', 'request_id': 'req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:23.142786 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d548ac70c184bc1883d130d11ccecb5', 'request_id': 'req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:23.217074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d548ac70c184bc1883d130d11ccecb5', 'request_id': 'req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:23.230849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] The token cache is being invalidate because project 2d548ac70c184bc1883d130d11ccecb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:29:23.231185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:23.231941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e79bb9be-b5cf-47e4-83ac-8fbe9186ec7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:29:23.233514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1182/2368] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:29:22 2022] DELETE /identity/v3/projects/2d548ac70c184bc1883d130d11ccecb5 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:29:23.238397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.238686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.238967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.265498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:23.266835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:23.341891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1187/2369] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:29:23.356737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.357091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.357382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.392336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:23.393546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:23.402168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1183/2370] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:23.406694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.416785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a098d-bdc2-43ca-a1ba-641ce1fa74ee None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.421837 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.422474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.422701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.422942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.423693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:29:23.424026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.425684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.438131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1188/2371] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:29:23.442668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.462499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e1a7-0861-41e0-bdb7-fcf69a34dedd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.470975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.473062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.473813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.474246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.477385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.479477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.581155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1184/2372] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:29:23 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:29:23.585942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.596766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ce43d-7eb7-47c7-abdb-d8cd509d4ea5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.663453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.664100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.664515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.664796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.665445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.667390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.802488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1189/2373] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:29:23 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:29:23.808990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.821732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e2edc-4471-42bb-a057-abc6336c2f7d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.825435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:23.828232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.829114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.829478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.835144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.835144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.835144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.844253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1185/2374] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:29:23.848534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.861438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed04b3-4362-4e58-8fdd-96b10899d20e admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.870437 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.874784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.874784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.874784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.874784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.874784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd834a-1bcf-4eb0-9627-4fa1eb863a73 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:23.887434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1190/2375] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:29:23.891720 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:23.894575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:23.905929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a7030-6619-4dc5-a5a5-3d4bf47a640c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:23.912033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:23.912995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:23.913490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:23.914425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] PATH_INFO: `/v3/projects/ac8e5530469c44cd8c4d962899f6ac5b/users/7aa5ec5bad5e41fcbdfd155bdcf6fd8a/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:23.954579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8e5530469c44cd8c4d962899f6ac5b, user_id=7aa5ec5bad5e41fcbdfd155bdcf6fd8a, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:23.957412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:24.043062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1186/2376] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:29:23 2022] PUT /identity/v3/projects/ac8e5530469c44cd8c4d962899f6ac5b/users/7aa5ec5bad5e41fcbdfd155bdcf6fd8a/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:29:24.054297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb5a7-6591-43ba-9e2e-40c0221f13ff None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:24.054585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb5a7-6591-43ba-9e2e-40c0221f13ff None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:24.058369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb5a7-6591-43ba-9e2e-40c0221f13ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:24.114989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-141cb5a7-6591-43ba-9e2e-40c0221f13ff None None] MFA Rules not processed for user `7aa5ec5bad5e41fcbdfd155bdcf6fd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:24.117158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-141cb5a7-6591-43ba-9e2e-40c0221f13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:24.228571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1191/2377] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:24.239727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:24.242163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:24.316353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4491b-d757-41d3-9347-51c47f9f3936 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:24.320711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:24.321320 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:24.321604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:24.321958 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:24.332756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:24.334246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa0ec5-fb79-4809-af05-44d2642db7e7 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:24.357891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1187/2378] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:29:25.586593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:25.838236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:26.359625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:26.360180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:26.360459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:26.386964 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] MFA Rules not processed for user `7aa5ec5bad5e41fcbdfd155bdcf6fd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:26.388225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:26.398891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1192/2379] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:26.738123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:26.823804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:27.128060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:28.915564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:29.053471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:29.887989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:30.012007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:30.966844 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:31.061435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:31.669445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:31.933455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:32.668601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:32.698766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:33.973742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:34.539726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:34.865855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:35.178058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:35.291177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:35.482221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:35.658006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:35.697747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:36.672931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:36.739826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:37.876719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:38.151869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:38.537994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:39.333533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.018594 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.264443 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:40.265065 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:40.265737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:40.293551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] MFA Rules not processed for user `7de2f8a8c6644220b9e1ed37a48b7d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:29:40.294775 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:40.354352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1188/2380] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:29:40.367333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.503294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.588864 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.754650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.819765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.870486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:40.893622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:41.270952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:41.451351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:41.731045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:41.848206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:41.890052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.018280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:42.020110 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:42.095086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.129045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe3d5-99c5-4eba-8623-11443622d477 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:42.139745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:42.140265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:42.140553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:42.140833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:42.142898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:42.157088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:42.158751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7aedb-70fd-4e42-8485-1deff48680be service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:42.169722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1193/2381] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:29:42.219077 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.355666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.440023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.470600 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:42.497274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:43.110414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:43.183342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:43.523784 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:43.887607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:44.441057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:54.144167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:54.526004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=21 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:29:59.150667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:29:59.152615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:59.216640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9f97d-35ab-44b0-a051-5cccbe55853d None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:29:59.220382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:29:59.221006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:29:59.221286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:29:59.221563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:29:59.223426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:29:59.293272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:29:59.294717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:29:59.306135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1189/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:30:00.292919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabc91-654a-4a42-8091-0927cf7018cf None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:00.293854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabc91-654a-4a42-8091-0927cf7018cf None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:00.294135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabc91-654a-4a42-8091-0927cf7018cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:00.318756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-dbdabc91-654a-4a42-8091-0927cf7018cf None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:00.320125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-dbdabc91-654a-4a42-8091-0927cf7018cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:00.384856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1194/2383] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:30:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:00.407930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:00.409302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:00.483853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566d63f-1cb0-4327-a78b-8b08a8d67ab4 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:00.488789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:00.489396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:00.489674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:00.490006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:00.498209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:00.499852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:00.508491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1190/2384] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:30:01.743255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e326ed-ec36-4ac1-8785-9b206ea2ffd9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:01.744595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e326ed-ec36-4ac1-8785-9b206ea2ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:01.745097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e326ed-ec36-4ac1-8785-9b206ea2ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:01.766790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c2e326ed-ec36-4ac1-8785-9b206ea2ffd9 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:01.768058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c2e326ed-ec36-4ac1-8785-9b206ea2ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:01.776271 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1195/2385] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:30:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:01.792022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:01.804678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b701-013e-48f0-9d74-8a4fce250e5d service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:01.812769 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:01.813543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:01.814037 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:01.814666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:01.824147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:01.825782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:01.836368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1191/2386] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:30:03.011912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78c17-79c9-453e-bec8-fe81b2fcbc03 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:03.012488 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78c17-79c9-453e-bec8-fe81b2fcbc03 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:03.013408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78c17-79c9-453e-bec8-fe81b2fcbc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:03.040672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-81f78c17-79c9-453e-bec8-fe81b2fcbc03 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:03.041910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-81f78c17-79c9-453e-bec8-fe81b2fcbc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:03.049720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1196/2387] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:03.063114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:03.074100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba8653-8a32-4fd5-a450-b4d24931d43b service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:03.079937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:03.080575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:03.080857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:03.081133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:03.090399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:03.092005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:03.100743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1192/2388] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 19:30:03.905574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:03.905866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:03.906150 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:03.926888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:03.928186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:03.936679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1197/2389] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:03.949608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:03.960321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebde972-e783-4b5b-a8c0-ab9f749cc980 service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:03.965354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:03.965946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:03.966230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:03.966506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:03.974400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:03.975988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c459c6-8fb3-4f6c-8b61-a7d9cf6d6967 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:03.980425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:03.984493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1193/2390] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:30:08.678853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:08.749109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8cf3-b27c-4a94-816c-06b05c0cb254 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:08.754295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:08.754818 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:08.755109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:08.755444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] PATH_INFO: `/v3/users/7aa5ec5bad5e41fcbdfd155bdcf6fd8a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:08.756783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa5ec5bad5e41fcbdfd155bdcf6fd8a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:08.758845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:08.995062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa5ec5bad5e41fcbdfd155bdcf6fd8a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.116211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa5ec5bad5e41fcbdfd155bdcf6fd8a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.116712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa5ec5bad5e41fcbdfd155bdcf6fd8a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.117616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa5ec5bad5e41fcbdfd155bdcf6fd8a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.125389 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa5ec5bad5e41fcbdfd155bdcf6fd8a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.144400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1198/2391] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:30:08 2022] DELETE /identity/v3/users/7aa5ec5bad5e41fcbdfd155bdcf6fd8a => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:09.149722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:09.150283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:09.150283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:09.171855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:09.173093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:09.247455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1194/2392] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:09.257623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:09.259562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:09.266525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:09.321825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f684a4-cfc3-4f3b-a411-1886d2699caa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:09.325620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:09.326333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:09.326679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:09.326962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:09.336550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:09.338006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6070a76-60d3-465d-807c-662594aa9bd0 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:09.349520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1199/2393] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:30:09.662642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:09.668915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:09.689615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d744b-2483-4956-be9c-eee0b3e2fccd None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:09.694978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:09.696797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:09.697188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:09.697488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] PATH_INFO: `/v3/projects/ac8e5530469c44cd8c4d962899f6ac5b` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:09.698787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8e5530469c44cd8c4d962899f6ac5b)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:09.701071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:09.872132 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8e5530469c44cd8c4d962899f6ac5b', 'request_id': 'req-1eef4bbb-0300-4e35-bb09-3901ee067f86'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:09.878607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8e5530469c44cd8c4d962899f6ac5b', 'request_id': 'req-1eef4bbb-0300-4e35-bb09-3901ee067f86'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:10.036142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8e5530469c44cd8c4d962899f6ac5b', 'request_id': 'req-1eef4bbb-0300-4e35-bb09-3901ee067f86'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:10.047670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] The token cache is being invalidate because project ac8e5530469c44cd8c4d962899f6ac5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:30:10.048140 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:10.048798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1eef4bbb-0300-4e35-bb09-3901ee067f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:10.050275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1195/2394] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:30:09 2022] DELETE /identity/v3/projects/ac8e5530469c44cd8c4d962899f6ac5b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:10.057400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.057869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.057957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.080258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:10.081461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:10.151010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1200/2395] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:10.158634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.159167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.159272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.182934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:10.183235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:10.191481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1196/2396] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:30:10.196045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.205538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e447e-6a80-494d-841f-51c12a394a69 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.210895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.211569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.211965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.212247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.213030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:30:10.213333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.214953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.228111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1201/2397] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:30:10.232624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.241940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f5db4-2317-4785-8f5c-8fc4037c7143 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.248326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.248953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.249236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.249517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.250150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.252078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.432327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1197/2398] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:30:10 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:30:10.436427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.447589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d2489-603b-4919-8ba3-1af1d9968e16 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.510957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.511571 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.511888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.512164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.512809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.514767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.688454 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1202/2399] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:30:10 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:30:10.692364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.703002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbac98-af1c-465b-9fad-940aeb16c745 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.709920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.710525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.710806 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.711569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.712315 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.713895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.724267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1198/2400] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:30:10.729047 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.739273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca95ed0-9551-44ef-ac6a-e78d3d6fa6e7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.744342 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.744944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.745230 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.745510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.746184 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.747839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a61960-3f84-4b64-a68a-6f863afcdbe4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.758650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1203/2401] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:30:10.763159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:10.774091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b1bcb-dd97-4865-8d9c-506c5ff44458 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:10.778813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:10.779466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.779785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.780066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] PATH_INFO: `/v3/projects/18b6ba58f0684cb699320fa8979047ae/users/e3624138edc34f89b9e92a97ac71faf4/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:10.795762 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b6ba58f0684cb699320fa8979047ae, user_id=e3624138edc34f89b9e92a97ac71faf4, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:10.798538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:10.979208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1199/2402] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:30:10 2022] PUT /identity/v3/projects/18b6ba58f0684cb699320fa8979047ae/users/e3624138edc34f89b9e92a97ac71faf4/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:10.984455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df233-f6f5-4026-85d2-a0ed978fc387 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:10.984924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df233-f6f5-4026-85d2-a0ed978fc387 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:10.985279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df233-f6f5-4026-85d2-a0ed978fc387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:11.004734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2c9df233-f6f5-4026-85d2-a0ed978fc387 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:11.005959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2c9df233-f6f5-4026-85d2-a0ed978fc387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:11.015302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:11.071985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1204/2403] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:11.082225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:11.083991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:11.106361 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:11.150492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c04910-16de-4b2a-883f-acf4d7c47a45 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:11.154200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:11.154871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:11.167004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:11.167004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:11.167004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:11.167004 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:11.177923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1200/2404] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:30:18.054646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.055267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.055267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.097410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] MFA Rules not processed for user `e3624138edc34f89b9e92a97ac71faf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:18.098645 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:18.188151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1205/2405] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:18.188729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.199875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0146e-a3cf-461f-a0ea-a681e4f7e968 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.204685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.205302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.205584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.205860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.206496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.208422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.354574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1201/2406] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:30:18 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:30:18.358782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.370276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3796f4-431c-4b4c-8478-4217d1945007 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.431331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.431977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.432262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.432542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.433301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.435244 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.563149 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1206/2407] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:30:18 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:30:18.567031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.580275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd755b-33fd-496b-aca4-b088a412e6db admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.585468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.586709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.587758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.587758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.587962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.589626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.601408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1202/2408] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:30:18.605647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.615882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b177483-86db-49dd-8c06-e0792b8fd196 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.620690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.621348 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.623254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.623254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] PATH_INFO: `/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d/users/ec4655613b524308925e7ebd56e1fd17/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.639243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6709b04e9b474ebe6d696dc4660b9d, user_id=ec4655613b524308925e7ebd56e1fd17, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.642267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.768178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1207/2409] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:30:18 2022] PUT /identity/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d/users/ec4655613b524308925e7ebd56e1fd17/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:18.772599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.786783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375bd23-3c66-4fae-a6a9-445a0060e000 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.857170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.857930 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.858218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.858492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.859142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.861029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390bf1b-548d-4a6d-a0cc-dd0ca5e863e2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.872756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1203/2410] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:30:18.876652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:18.888166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f65331-8512-445f-853f-c9a00892ef4d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:18.902106 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:18.902793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:18.903073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:18.903404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] PATH_INFO: `/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d/users/ec4655613b524308925e7ebd56e1fd17/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:18.905439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6709b04e9b474ebe6d696dc4660b9d, user_id=ec4655613b524308925e7ebd56e1fd17, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:18.908261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:18.911772 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:18.921994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1208/2411] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:30:18 2022] PUT /identity/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d/users/ec4655613b524308925e7ebd56e1fd17/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:30:22.342536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe0c45-d500-4abb-b6c7-c0ebda7ddd68 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:22.344053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe0c45-d500-4abb-b6c7-c0ebda7ddd68 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:22.344540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe0c45-d500-4abb-b6c7-c0ebda7ddd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:22.369279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9bbe0c45-d500-4abb-b6c7-c0ebda7ddd68 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:22.371062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9bbe0c45-d500-4abb-b6c7-c0ebda7ddd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:22.446696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1204/2412] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:22.461144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:22.463223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:22.562449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ae383-6fd0-44c1-b909-f86381797efa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:22.568292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:22.568930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:22.569217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:22.569489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:22.578997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:22.580669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:22.591533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1209/2413] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:30:28.313314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3b968-f891-4c9b-8b49-4c3bdc522851 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:28.315935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3b968-f891-4c9b-8b49-4c3bdc522851 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:28.317779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3b968-f891-4c9b-8b49-4c3bdc522851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:28.424533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e1a3b968-f891-4c9b-8b49-4c3bdc522851 None None] MFA Rules not processed for user `ec4655613b524308925e7ebd56e1fd17`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:28.426898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e1a3b968-f891-4c9b-8b49-4c3bdc522851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:28.491181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:28.589893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1205/2414] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:28.615687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:28.617934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:28.780380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416496c-6781-4ac3-8482-0afa090a3d0b service swift] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:28.801070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:28.801896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:28.802297 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:28.802715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:28.955628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:28.957669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:28.973308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1210/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:30:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:30:38.673639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=20 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:38.967545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:51.161309 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ef275-0edd-4df3-8a6e-3e5a6f7fa311 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:51.161900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ef275-0edd-4df3-8a6e-3e5a6f7fa311 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:51.161900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ef275-0edd-4df3-8a6e-3e5a6f7fa311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:51.190684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-eb0ef275-0edd-4df3-8a6e-3e5a6f7fa311 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:51.192319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-eb0ef275-0edd-4df3-8a6e-3e5a6f7fa311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:51.206150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1206/2416] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:30:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:51.219561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:51.233069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18b54-7060-45e1-b685-32b47d7ac030 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:51.238694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:51.242526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:51.242526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:51.242526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:51.255989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:51.259045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:51.275511 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1211/2417] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:30:53.569130 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:53.569623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:53.569718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:53.595357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] MFA Rules not processed for user `0e37c8ca70ad4b6e8430a71d0f5e564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:53.596638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:53.656692 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1207/2418] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:55.191096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:55.192629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:55.265498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65aa6-f5ae-4b40-881a-129b57b19627 service swift] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:55.270123 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:55.270755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:55.271040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:55.271316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] PATH_INFO: `/v3/users/7de2f8a8c6644220b9e1ed37a48b7d9a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:55.272480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de2f8a8c6644220b9e1ed37a48b7d9a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:55.274904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:55.484798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de2f8a8c6644220b9e1ed37a48b7d9a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:55.606695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de2f8a8c6644220b9e1ed37a48b7d9a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:55.607343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de2f8a8c6644220b9e1ed37a48b7d9a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:55.607991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de2f8a8c6644220b9e1ed37a48b7d9a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:55.616020 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-ca285906-7714-4329-bf4a-060aa02344a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de2f8a8c6644220b9e1ed37a48b7d9a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:55.635750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1212/2419] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:30:55 2022] DELETE /identity/v3/users/7de2f8a8c6644220b9e1ed37a48b7d9a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:55.906381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:55.919015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994375-7ef5-43c5-bfca-e3bd140e7dcb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:55.994783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:55.999238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:55.999238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:55.999238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] PATH_INFO: `/v3/projects/cf90b9345ae84e26b58e9238da0dac2e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:55.999238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf90b9345ae84e26b58e9238da0dac2e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:55.999238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.168341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf90b9345ae84e26b58e9238da0dac2e', 'request_id': 'req-057899a9-1031-489f-bf1e-899259e70c0f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:56.175323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf90b9345ae84e26b58e9238da0dac2e', 'request_id': 'req-057899a9-1031-489f-bf1e-899259e70c0f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:56.258475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf90b9345ae84e26b58e9238da0dac2e', 'request_id': 'req-057899a9-1031-489f-bf1e-899259e70c0f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:56.271390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] The token cache is being invalidate because project cf90b9345ae84e26b58e9238da0dac2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:30:56.272240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:56.273064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-057899a9-1031-489f-bf1e-899259e70c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:30:56.274679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1208/2420] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:30:55 2022] DELETE /identity/v3/projects/cf90b9345ae84e26b58e9238da0dac2e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:30:56.279864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.280238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.280409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.301301 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:56.302581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:56.364822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1213/2421] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:56.431164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.431506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.431829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.453078 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:56.454280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:56.462124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1209/2422] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:56.466612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.476771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e54a-09fe-4862-9e8f-3c66107650d3 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.482030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.482630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.482919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.483201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.483886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:30:56.484193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.485811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.487364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:56.501095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1214/2423] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:30:56.504200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.513212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f84c5-38bb-4d6a-a3fe-1ac2caf9090a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.517896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.518488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.518782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.519043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.520951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.521739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.613625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:56.621422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1210/2424] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:30:56 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:30:56.626089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.636254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ea48-f1ea-4c17-aafb-9c0209580709 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.700007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.700618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.700909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.701191 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.701835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.703875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.855720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1215/2425] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:30:56 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:30:56.859639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.870481 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc137e1c-e052-442b-8395-ec3510105a79 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.876390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.877002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.877284 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.877556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.878211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.879942 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.890552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1211/2426] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:30:56.894878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.905861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fdc13-8c71-4b93-bad2-c736d4588cf2 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.911329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.912195 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.912512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.912794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.913449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.915628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e57f1d-7305-4b0d-a377-78e3d8ecfec8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:56.932605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1216/2427] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:30:56.933338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:56.943704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739d5e-503e-4144-9577-b2507bee1491 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:56.948614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:56.949387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:56.949750 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:56.950091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] PATH_INFO: `/v3/projects/ee88123fe2e34b02a00baeb6f74869aa/users/8c76463423664f649bd018d853010cad/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:56.967517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee88123fe2e34b02a00baeb6f74869aa, user_id=8c76463423664f649bd018d853010cad, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:56.970636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:57.066832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1212/2428] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:30:56 2022] PUT /identity/v3/projects/ee88123fe2e34b02a00baeb6f74869aa/users/8c76463423664f649bd018d853010cad/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:30:57.072233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fa55-41e8-439e-bfcd-34b924475931 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:57.072514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fa55-41e8-439e-bfcd-34b924475931 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:57.072800 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fa55-41e8-439e-bfcd-34b924475931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:57.093216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-fd24fa55-41e8-439e-bfcd-34b924475931 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:57.094429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fd24fa55-41e8-439e-bfcd-34b924475931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:57.155476 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1217/2429] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:57.165469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:57.167177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:57.227962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663f7cc-3d34-45af-9be0-7e09b56b53a3 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:57.231609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:57.232247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:57.232527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:57.232800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:57.242727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:57.244988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddb28e-c65d-4dd6-bb7e-9f9c09aef06d service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:57.255080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1213/2430] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:30:58.022240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:58.022955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:58.022955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:58.049663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] MFA Rules not processed for user `e3624138edc34f89b9e92a97ac71faf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:58.051231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:58.132375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1218/2431] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:59.645221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318b9b-40c0-4b0f-a2db-1a2ab40adf8d None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:59.645794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318b9b-40c0-4b0f-a2db-1a2ab40adf8d None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:59.646424 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318b9b-40c0-4b0f-a2db-1a2ab40adf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:59.692059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-85318b9b-40c0-4b0f-a2db-1a2ab40adf8d None None] MFA Rules not processed for user `8c76463423664f649bd018d853010cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:30:59.700632 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-85318b9b-40c0-4b0f-a2db-1a2ab40adf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:59.788640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1214/2432] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:30:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:30:59.799957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:30:59.802003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:30:59.828386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:30:59.874662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71333628-ce3e-4f2f-b9cf-0e6df38d8282 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:30:59.878335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:30:59.878947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:30:59.879232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:30:59.879516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:30:59.888724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:30:59.890168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d4639-f084-429f-809e-ad219a09510b service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:30:59.899055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1219/2433] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:30:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:31:01.817918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:01.818463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:01.818581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:01.839107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] MFA Rules not processed for user `8c76463423664f649bd018d853010cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:01.844984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:01.847550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1215/2434] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:10.504061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=18 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:11.833686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:24.425189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971bf7-9208-413c-8cb7-a0cb7b399d65 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:24.425753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971bf7-9208-413c-8cb7-a0cb7b399d65 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:24.425753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971bf7-9208-413c-8cb7-a0cb7b399d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:24.445530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-5f971bf7-9208-413c-8cb7-a0cb7b399d65 None None] MFA Rules not processed for user `1d7c40b5232e46119434aeeb8a6ad4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:24.446754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5f971bf7-9208-413c-8cb7-a0cb7b399d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:24.501541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1220/2435] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:33.600727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:33.602118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:33.614615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a583-dc41-4a4a-8866-a54ac0dfcac5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:33.618145 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:33.618735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:33.619014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:33.619283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:33.621126 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:33.697470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:33.699002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:33.711907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1216/2436] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:31:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:31:34.495158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=25 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:40.555519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:40.556089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:40.556177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:40.576931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:40.578011 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:40.632617 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1221/2437] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:31:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:40.656099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:40.657561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:40.727341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb61f4-8818-4097-be47-0ee95875caf8 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:40.732348 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:40.733495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:40.733495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:40.733741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:40.742230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:40.743872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:40.752038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1217/2438] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:31:49.873469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:49.875142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:49.890088 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ced0e-be3d-4bed-913d-1012e31b2d47 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:49.894798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:49.895435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:49.895816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:49.896060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] PATH_INFO: `/v3/users/fd8d3fe4be354995b1698b9783ce6c65` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:49.897186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8d3fe4be354995b1698b9783ce6c65)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:49.899116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:50.110952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8d3fe4be354995b1698b9783ce6c65'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.189251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8d3fe4be354995b1698b9783ce6c65'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.189873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8d3fe4be354995b1698b9783ce6c65'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.190475 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8d3fe4be354995b1698b9783ce6c65'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.198219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8d3fe4be354995b1698b9783ce6c65'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.216955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1222/2439] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:31:49 2022] DELETE /identity/v3/users/fd8d3fe4be354995b1698b9783ce6c65 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:50.488516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:50.501157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e185c1-4ba4-4ba0-a443-6f0c13bbc3be service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:50.567222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:50.567894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:50.568195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:50.568529 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] PATH_INFO: `/v3/projects/4fd3f2da26af4d3db4c6db4ceb851558` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:50.569945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd3f2da26af4d3db4c6db4ceb851558)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:50.571765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:50.590861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:50.760336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd3f2da26af4d3db4c6db4ceb851558', 'request_id': 'req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.768766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd3f2da26af4d3db4c6db4ceb851558', 'request_id': 'req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.875522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd3f2da26af4d3db4c6db4ceb851558', 'request_id': 'req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.887637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] The token cache is being invalidate because project 4fd3f2da26af4d3db4c6db4ceb851558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:31:50.888149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.888878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:50.890499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1218/2440] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:31:50 2022] DELETE /identity/v3/projects/4fd3f2da26af4d3db4c6db4ceb851558 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:50.894715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:50.896214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:50.916181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54a972-79d2-49dc-ba19-8149a79168c3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:50.983422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:50.984010 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:50.984295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:50.984574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] PATH_INFO: `/v3/users/8639718ca51d40d7850cb4212d69c214` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:50.985714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8639718ca51d40d7850cb4212d69c214)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:50.987731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:51.009065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:51.220428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8639718ca51d40d7850cb4212d69c214'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.298103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8639718ca51d40d7850cb4212d69c214'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.298742 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8639718ca51d40d7850cb4212d69c214'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.299366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8639718ca51d40d7850cb4212d69c214'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.307050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-7c6c30dd-902d-46b2-91e3-8a6c93348b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8639718ca51d40d7850cb4212d69c214'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.326547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1223/2441] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:31:50 2022] DELETE /identity/v3/users/8639718ca51d40d7850cb4212d69c214 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:51.569240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:51.570676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:51.584567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a18c4-0f24-4c6a-ac67-7ef310a39c29 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:51.644972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:51.645605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:51.645891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:51.646220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] PATH_INFO: `/v3/projects/52ab88f388b84b5488a950739c0710d0` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:51.647337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ab88f388b84b5488a950739c0710d0)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:51.649327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:51.773477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ab88f388b84b5488a950739c0710d0', 'request_id': 'req-80e0fded-7290-4137-8891-7139362f9df0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.779721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ab88f388b84b5488a950739c0710d0', 'request_id': 'req-80e0fded-7290-4137-8891-7139362f9df0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.909604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ab88f388b84b5488a950739c0710d0', 'request_id': 'req-80e0fded-7290-4137-8891-7139362f9df0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.922366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] The token cache is being invalidate because project 52ab88f388b84b5488a950739c0710d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:31:51.928733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.928733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80e0fded-7290-4137-8891-7139362f9df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:31:51.928733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1219/2442] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:31:51 2022] DELETE /identity/v3/projects/52ab88f388b84b5488a950739c0710d0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:51.930320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:51.930809 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:51.930906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:51.957032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:51.958265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:52.024527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1224/2443] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:31:52.032731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.033019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.033299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.053823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:52.055093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:52.064201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1220/2444] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:52.068702 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.078724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618397e-f372-4b98-804b-1aba193fde1c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.083717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.084334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.084625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.084900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.085556 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:31:52.085863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.087464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.099967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1225/2445] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:31:52.104526 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.114898 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f8f8-c5ba-4e59-8722-63a0bd1dc017 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.119664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.120414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.120705 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.120977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.121621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.123516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.231156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1221/2446] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:31:52 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:31:52.236091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.246220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102baa-9ab1-4cb5-acb1-d9848ae3f7fb admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.306085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.306660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.306968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.307240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.307909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.310120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.407722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1226/2447] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:31:52 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:31:52.411877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.423092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74931-5c6d-4abc-a76a-1b7fa86afe2e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.429044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.429651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.429982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.430475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.431238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.432987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.434955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:52.444250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1222/2448] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:31:52.451520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.461595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f53c0-439f-4a93-bc32-fe45fe41f426 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.467036 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.467636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.467935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.468204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.468920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.470510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbeea95-7784-4b66-8659-fec47b6d3121 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.482453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1227/2449] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:31:52.485650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.496233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477da92-0c4e-4476-9ac0-12997d3efc41 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.501442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.502106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.502439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.502694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] PATH_INFO: `/v3/projects/a7bbec449c0142f0b910956bacedbe5e/users/b638c6d935e64be6bce4ecc1fb8f26ec/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.517957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bbec449c0142f0b910956bacedbe5e, user_id=b638c6d935e64be6bce4ecc1fb8f26ec, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.520636 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.584600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1223/2450] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:31:52 2022] PUT /identity/v3/projects/a7bbec449c0142f0b910956bacedbe5e/users/b638c6d935e64be6bce4ecc1fb8f26ec/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:52.589755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.590044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.590319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.598214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:52.610716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:52.611954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:52.672990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1228/2451] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:52.682709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:52.684793 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:52.745017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8afa-7a70-4142-8369-4f944028d41f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:52.748792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:52.749405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:52.749685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:52.749967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:52.759579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:52.761047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9d7d5-500a-4d1b-b62c-8354f0b6be58 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:52.771432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1224/2452] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:31:57.334586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:57.336749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:57.427642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b841eb8-1e40-401a-bbe8-d53d8247b3e2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:57.435623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:57.435623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:57.435623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:57.435623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:57.437569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:57.502377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:57.504079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:57.527416 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1229/2453] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:31:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:31:58.659988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:58.660458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:58.660558 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:58.691768 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] MFA Rules not processed for user `b638c6d935e64be6bce4ecc1fb8f26ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:31:58.692764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:31:58.792409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1225/2454] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:31:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:31:58.797857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:58.814015 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87dd0f-4c79-4c1f-815c-0a1d79137a36 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:58.819205 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:58.819985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:58.820327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:58.820669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:58.821378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:58.823530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:58.978276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1230/2455] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:31:58 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:31:58.980397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:58.992425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dfa66-32fd-43d4-b38b-21a14cfa8ec3 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:59.068036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:59.068670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:59.069059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:59.069280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:59.070017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:59.071964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:59.085079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:59.158095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1226/2456] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:31:58 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:31:59.164443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:59.176316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768f34-dfc6-4276-b06b-9156f09620db admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:59.182307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:59.182307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:59.182590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:59.182996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:59.183677 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:59.187869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:59.202038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1231/2457] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:31:59.206722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:59.222338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12dbc-059d-4dc2-85d3-a24cd4aab3cd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:59.227100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:59.227972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:59.228268 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:59.228548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] PATH_INFO: `/v3/projects/19f7aac894e942ce8e3eb07d97871f1b/users/a598e93e631b4cbcaac7a047fb1d5caa/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:59.249881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7aac894e942ce8e3eb07d97871f1b, user_id=a598e93e631b4cbcaac7a047fb1d5caa, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:59.253177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:59.323175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1227/2458] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:31:59 2022] PUT /identity/v3/projects/19f7aac894e942ce8e3eb07d97871f1b/users/a598e93e631b4cbcaac7a047fb1d5caa/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:31:59.327916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:59.339579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f01fa-fb59-4f25-93ef-36a75598f7d5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:59.343873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:31:59.445781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:59.452792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:59.452792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:59.452792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:59.452792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:59.452792 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:59.487255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1232/2459] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:31:59.489895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:31:59.506128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed002ee-8256-4338-9c6c-1b2ac1bc9b23 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:31:59.514059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:31:59.519028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:31:59.519028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:31:59.519028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] PATH_INFO: `/v3/projects/19f7aac894e942ce8e3eb07d97871f1b/users/a598e93e631b4cbcaac7a047fb1d5caa/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:31:59.522923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7aac894e942ce8e3eb07d97871f1b, user_id=a598e93e631b4cbcaac7a047fb1d5caa, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:31:59.527137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426333-65c0-49dc-bb2f-273005d82200 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:31:59.549368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1228/2460] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:31:59 2022] PUT /identity/v3/projects/19f7aac894e942ce8e3eb07d97871f1b/users/a598e93e631b4cbcaac7a047fb1d5caa/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:05.811575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:05.815262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:05.971381 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d167-a7a1-4462-bd04-5962888f20b1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:06.065060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:06.066066 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:06.066527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:06.066822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:06.068893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:06.083464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:06.087578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:06.098441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1233/2461] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:07.741434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:07.742009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:07.742104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:07.778383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] MFA Rules not processed for user `a598e93e631b4cbcaac7a047fb1d5caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:07.780291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:07.881921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1229/2462] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:07.900504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:07.953445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:08.004967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f07e1-ff1e-4604-b4c4-e37199c9a250 service placement] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:08.016385 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:08.017128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:08.017520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:08.017890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:08.117595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:08.127295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:08.139473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1234/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:11.516643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:11.518436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:11.620483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8ad6a-2d18-4837-8eb3-63e604c826e5 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:11.627161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:11.628666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:11.629286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:11.629649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:11.634000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:11.803469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:11.806974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:11.809070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ac0ec-b6b3-49a5-8d99-40d7107abeb6 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:11.823439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1230/2464] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:32:12.065370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:12.067548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:12.153651 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0b3db-8738-43dc-bfb1-310d456fa7d6 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:12.161118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:12.162019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:12.162487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:12.163390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] PATH_INFO: `/v3/users/e3624138edc34f89b9e92a97ac71faf4` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:12.165168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3624138edc34f89b9e92a97ac71faf4)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:12.168387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:12.336751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:12.336751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:12.336751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:12.356483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] MFA Rules not processed for user `b638c6d935e64be6bce4ecc1fb8f26ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:12.358298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:12.373217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1231/2465] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:12.488509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3624138edc34f89b9e92a97ac71faf4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:12.630878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:12.644342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a19d-a133-4406-806a-f94f2a44994a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:12.651063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:12.651719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:12.652005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:12.652281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:12.654899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:12.658566 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3624138edc34f89b9e92a97ac71faf4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:12.659370 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3624138edc34f89b9e92a97ac71faf4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:12.660065 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3624138edc34f89b9e92a97ac71faf4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:12.670027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:12.675307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:12.681436 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3624138edc34f89b9e92a97ac71faf4'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:12.685405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1232/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:32:12.714853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:12.734068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1235/2467] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:32:12 2022] DELETE /identity/v3/users/e3624138edc34f89b9e92a97ac71faf4 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:13.155956 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:13.169549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816b7ea-8960-4e60-a923-eba96eebaa8b service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:13.247184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:13.248727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:13.249013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:13.249285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] PATH_INFO: `/v3/projects/18b6ba58f0684cb699320fa8979047ae` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:13.250422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b6ba58f0684cb699320fa8979047ae)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:13.252690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:13.585311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b6ba58f0684cb699320fa8979047ae', 'request_id': 'req-7255d473-7017-43a2-9fd1-670e4cbcbfb6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:13.596459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b6ba58f0684cb699320fa8979047ae', 'request_id': 'req-7255d473-7017-43a2-9fd1-670e4cbcbfb6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:13.733157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b6ba58f0684cb699320fa8979047ae', 'request_id': 'req-7255d473-7017-43a2-9fd1-670e4cbcbfb6'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:13.749801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] The token cache is being invalidate because project 18b6ba58f0684cb699320fa8979047ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:32:13.749801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:13.750022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:13.754251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1233/2468] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:32:13 2022] DELETE /identity/v3/projects/18b6ba58f0684cb699320fa8979047ae => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:13.755489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:13.757006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:13.773407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c25f6-19ae-4a65-b0ca-239faedecefc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:13.842422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:13.843207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:13.843584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:13.844027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] PATH_INFO: `/v3/users/ec4655613b524308925e7ebd56e1fd17` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:13.845722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4655613b524308925e7ebd56e1fd17)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:13.847951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:14.055151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4655613b524308925e7ebd56e1fd17'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.149507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4655613b524308925e7ebd56e1fd17'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.150406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4655613b524308925e7ebd56e1fd17'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.151386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4655613b524308925e7ebd56e1fd17'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.164802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-899d4d60-9037-46f5-84f3-133285849018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4655613b524308925e7ebd56e1fd17'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.189108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1236/2469] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:32:13 2022] DELETE /identity/v3/users/ec4655613b524308925e7ebd56e1fd17 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:14.512663 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:14.514623 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:14.531473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255d473-7017-43a2-9fd1-670e4cbcbfb6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:14.605859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:14.606527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:14.606837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:14.607114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] PATH_INFO: `/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:14.608357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6709b04e9b474ebe6d696dc4660b9d)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:14.610306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:14.621008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:14.780453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6709b04e9b474ebe6d696dc4660b9d', 'request_id': 'req-a9f55a49-09c2-44ce-ac94-5864094febcd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.787626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6709b04e9b474ebe6d696dc4660b9d', 'request_id': 'req-a9f55a49-09c2-44ce-ac94-5864094febcd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.864568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6709b04e9b474ebe6d696dc4660b9d', 'request_id': 'req-a9f55a49-09c2-44ce-ac94-5864094febcd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.882215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] The token cache is being invalidate because project ac6709b04e9b474ebe6d696dc4660b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:32:14.882751 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.884028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:14.895678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1234/2470] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:32:14 2022] DELETE /identity/v3/projects/ac6709b04e9b474ebe6d696dc4660b9d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:24.583898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:25.219530 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:29.740706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151036e0-ada4-4633-8750-8511b706c75b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:29.741132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151036e0-ada4-4633-8750-8511b706c75b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:29.741397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151036e0-ada4-4633-8750-8511b706c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:29.762757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-151036e0-ada4-4633-8750-8511b706c75b None None] MFA Rules not processed for user `0e37c8ca70ad4b6e8430a71d0f5e564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:29.763796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-151036e0-ada4-4633-8750-8511b706c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:29.825371 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1237/2471] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:30.147829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:30.149367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:30.213755 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55a49-09c2-44ce-ac94-5864094febcd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:30.218201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:30.218816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:30.219211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:30.219493 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:30.221285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:30.231732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:30.233160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c2679-a0c0-4015-902e-81d1778fd4af service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:30.242022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1235/2472] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:30.647089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151036e0-ada4-4633-8750-8511b706c75b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:30.649433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-151036e0-ada4-4633-8750-8511b706c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:30.731547 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849006-cff1-483f-aa76-77d9cbf84453 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:30.731931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849006-cff1-483f-aa76-77d9cbf84453 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:30.732372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849006-cff1-483f-aa76-77d9cbf84453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:30.757483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151036e0-ada4-4633-8750-8511b706c75b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:30.759265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-61849006-cff1-483f-aa76-77d9cbf84453 None None] MFA Rules not processed for user `8c76463423664f649bd018d853010cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:30.760683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-61849006-cff1-483f-aa76-77d9cbf84453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:30.762915 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:30.772142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:30.772142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:30.772142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:30.772142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:30.778926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:30.781166 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:30.791773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1238/2473] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:30.829642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1236/2474] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:32:35.197353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:35.208551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3a57-7b5c-4e87-b7ce-2560a9a31419 service placement] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:35.212337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:35.212950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:35.213241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:35.213523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:35.215567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:35.226788 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:35.228363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:35.237812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1239/2475] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:36.718056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:36.718603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:36.718603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:36.739551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] MFA Rules not processed for user `cfbdd49e2fa64b428d957d9dcf1d6846`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:36.740943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:36.839576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1237/2476] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:39.942489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:39.944737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:40.043697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24064d8-a094-4834-b40b-1c82efabd9cd service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:40.056592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:40.057383 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:40.057813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:40.058142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:40.061769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:40.075911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:40.078572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:40.091563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1240/2477] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:42.646333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:42.662253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2af6-1804-4261-b649-13a0f91e480c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:42.667989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:42.668674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:42.669028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:42.669373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:42.671215 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:42.684064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:42.685966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:42.700306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1238/2478] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:48.999415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:49.001507 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:49.018812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90d9e4-cddd-48c2-a9ad-34a00445134f service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:49.023384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:49.027484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:49.027484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:49.027484 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:49.030712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:49.037864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:49.107262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:49.108877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:49.132561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1241/2479] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:52.693906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=82 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:52.992693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:52.994635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:53.011680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414523-963f-4241-b349-8090cd93bd2e service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:53.021389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:53.022339 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:53.022785 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:53.023245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:53.025314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:53.040450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:53.042291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:53.065797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1239/2480] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:53.438504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:53.440029 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:53.453120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e06572-5e51-4d72-8b09-2ad640676abc service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:53.459129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:53.460038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:53.460318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:53.460650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] PATH_INFO: `/v3/users/1d7c40b5232e46119434aeeb8a6ad4d0` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:53.461954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7c40b5232e46119434aeeb8a6ad4d0)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:53.464157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:53.707538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7c40b5232e46119434aeeb8a6ad4d0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:53.779545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7c40b5232e46119434aeeb8a6ad4d0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:53.782369 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7c40b5232e46119434aeeb8a6ad4d0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:53.783400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7c40b5232e46119434aeeb8a6ad4d0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:53.800179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-998aefbc-15e9-4a5d-bbfa-1d43cd06488e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7c40b5232e46119434aeeb8a6ad4d0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:53.826214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1242/2481] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:32:53 2022] DELETE /identity/v3/users/1d7c40b5232e46119434aeeb8a6ad4d0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:54.177448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:54.191294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01c2cc-fd9f-4902-8f67-44c10399bd5b service placement] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:54.260981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:54.261606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.261885 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.262157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] PATH_INFO: `/v3/projects/016f968319fd49d98506030c0bee6021` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.263602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016f968319fd49d98506030c0bee6021)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:54.265645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:54.437149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016f968319fd49d98506030c0bee6021', 'request_id': 'req-a7a80208-9b8e-4651-a192-7f64b1dd6934'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:54.443537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016f968319fd49d98506030c0bee6021', 'request_id': 'req-a7a80208-9b8e-4651-a192-7f64b1dd6934'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:54.554950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016f968319fd49d98506030c0bee6021', 'request_id': 'req-a7a80208-9b8e-4651-a192-7f64b1dd6934'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:54.567024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] The token cache is being invalidate because project 016f968319fd49d98506030c0bee6021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:32:54.567314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:54.568067 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a7a80208-9b8e-4651-a192-7f64b1dd6934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:32:54.569476 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1240/2482] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:32:54 2022] DELETE /identity/v3/projects/016f968319fd49d98506030c0bee6021 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:54.574621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.575144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.575346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.595491 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:54.596757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:54.666396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1243/2483] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:54.675486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.675817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.676100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.696559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:54.697794 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:54.706330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1241/2484] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:54.710582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:54.720856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a77b28-b073-4122-a7ae-e75baa50ac4c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:54.725456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:54.726048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.726333 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.726611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.727256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:32:54.727561 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:54.729224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:54.737802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:54.741061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1244/2485] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:32:54.745423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:54.754803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3fb6b-b40e-4049-8c9b-43de48cdccbe None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:54.759502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:54.760122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.760405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.760678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.761324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:54.763195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:54.870294 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1242/2486] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:32:54 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:32:54.870509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:54.880641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909e10-f351-4971-b543-8f4b925377c0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:54.950155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:54.950770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:54.951052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:54.951325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:54.952654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:54.954880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:55.025650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1245/2487] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:32:54 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:32:55.030817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:55.042974 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826cdb-b992-4a61-a6f5-95e368f367dc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:55.048590 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:55.049218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:55.049509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:55.049818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:55.050509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:55.052175 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:55.064511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1243/2488] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:32:55.068650 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:55.079963 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7100ef-b7e9-4733-9b87-58298736e31b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:55.084681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:55.085306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:55.085590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:55.085868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:55.086522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:55.088189 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c64b4-5b9b-4e0f-a01a-ec4b743033a9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:55.102890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1246/2489] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:32:55.104246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:55.114760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77b5b-45c7-4e1f-bbf6-1fa1572f3dbc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:55.119596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:55.120288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:55.120567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:55.120869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] PATH_INFO: `/v3/projects/427b697da1034408811e994e22d4f592/users/250673e17f40455ca2c5e0036f5a9e3c/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:55.136499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427b697da1034408811e994e22d4f592, user_id=250673e17f40455ca2c5e0036f5a9e3c, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:55.139174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:55.219364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1244/2490] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:32:55 2022] PUT /identity/v3/projects/427b697da1034408811e994e22d4f592/users/250673e17f40455ca2c5e0036f5a9e3c/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:32:55.225062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:55.225062 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:55.226203 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:55.247753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:32:55.249190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:55.323538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1247/2491] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:32:55.336019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:55.340894 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:55.359353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:32:55.412744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ce366-d181-42e5-89ee-f7034a335301 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:55.416451 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:55.417059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:55.417345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:55.417624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:55.427387 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:55.428873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ecf2c-5d57-4a7d-8e3f-5cc14532709b service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:55.439722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1245/2492] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:32:58.901310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:32:58.917394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2010e9-c4bc-4dc8-bb14-4141cb790669 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:32:58.937368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:32:58.937368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:32:58.937368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:32:58.937368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:32:58.951712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:32:59.093426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:32:59.095882 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:32:59.114855 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1248/2493] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:32:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:33:01.650803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca3d7-39b1-4e66-a5e9-0829e58f2323 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:01.650803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca3d7-39b1-4e66-a5e9-0829e58f2323 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:01.650803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca3d7-39b1-4e66-a5e9-0829e58f2323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:01.684325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6deca3d7-39b1-4e66-a5e9-0829e58f2323 None None] MFA Rules not processed for user `250673e17f40455ca2c5e0036f5a9e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:33:01.685923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6deca3d7-39b1-4e66-a5e9-0829e58f2323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:01.776328 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1246/2494] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:33:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:33:01.784857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:33:01.786093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:01.815405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:33:01.899078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a0d8d-a4af-4e68-8368-2106f0d0f0ae service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:33:01.908463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:33:01.909201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:01.909547 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:01.909900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:01.920238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:33:01.921310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:33:01.939248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1249/2495] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:33:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:33:05.168626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:05.169157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:05.172238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:05.206823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] MFA Rules not processed for user `250673e17f40455ca2c5e0036f5a9e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:33:05.208219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:05.220106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1247/2496] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:33:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:33:11.931874 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=28 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:33:15.198415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=55 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:33:47.096941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:33:47.096941 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:47.109401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec925c60-5c24-4c1b-b058-9f1f8d2f9ca4 service nova] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:33:47.114642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:33:47.115378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:47.115688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:47.115976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] PATH_INFO: `/v3/users/8c76463423664f649bd018d853010cad` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:47.117134 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c76463423664f649bd018d853010cad)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:33:47.119070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:33:47.181571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:33:47.195410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48973-8fdf-484c-bd6a-cb0ff8d6c691 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:33:47.200934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:33:47.201517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:47.201797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:47.202072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:47.203799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:47.265012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:33:47.266591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:33:47.275607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1248/2497] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:33:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:33:47.469304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c76463423664f649bd018d853010cad'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:47.589750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c76463423664f649bd018d853010cad'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:47.590884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c76463423664f649bd018d853010cad'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:47.592176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c76463423664f649bd018d853010cad'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:47.602431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c76463423664f649bd018d853010cad'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:47.623731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1250/2498] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:33:47 2022] DELETE /identity/v3/users/8c76463423664f649bd018d853010cad => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:33:47.963719 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:33:47.976063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf965-8b72-4de2-98b0-573f6802dad3 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:33:48.055371 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:33:48.056449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:48.056919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:48.057453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] PATH_INFO: `/v3/projects/ee88123fe2e34b02a00baeb6f74869aa` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:48.058813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee88123fe2e34b02a00baeb6f74869aa)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:33:48.069469 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:33:48.234603 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee88123fe2e34b02a00baeb6f74869aa', 'request_id': 'req-7a658218-c0b0-4a84-a94a-15563b331593'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:48.242305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee88123fe2e34b02a00baeb6f74869aa', 'request_id': 'req-7a658218-c0b0-4a84-a94a-15563b331593'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:48.360128 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee88123fe2e34b02a00baeb6f74869aa', 'request_id': 'req-7a658218-c0b0-4a84-a94a-15563b331593'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:48.377031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] The token cache is being invalidate because project ee88123fe2e34b02a00baeb6f74869aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:33:48.377031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:48.377329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:33:48.379329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1249/2499] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:33:47 2022] DELETE /identity/v3/projects/ee88123fe2e34b02a00baeb6f74869aa => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:33:56.880592 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:33:56.882068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:56.944730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72f0fc-f3ed-4fbe-9f32-0fbfe8c292f0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:33:56.996035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:33:56.996714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:33:56.997000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:33:56.997276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:33:56.999526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:33:57.042892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:33:57.064175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:33:57.065685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:33:57.075408 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1251/2500] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:33:58.371167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:01.137899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:01.140024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:01.159367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a658218-c0b0-4a84-a94a-15563b331593 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:01.170213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:01.170855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:01.171327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:01.171606 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] PATH_INFO: `/v3/users/0e37c8ca70ad4b6e8430a71d0f5e564f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:01.172876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e37c8ca70ad4b6e8430a71d0f5e564f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:01.174800 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:01.380816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e37c8ca70ad4b6e8430a71d0f5e564f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:01.605593 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e37c8ca70ad4b6e8430a71d0f5e564f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:01.606255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e37c8ca70ad4b6e8430a71d0f5e564f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:01.606909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e37c8ca70ad4b6e8430a71d0f5e564f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:01.620232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-898a9449-8d84-4ab7-b054-df43eefbce64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e37c8ca70ad4b6e8430a71d0f5e564f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:01.633155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1250/2501] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:34:01 2022] DELETE /identity/v3/users/0e37c8ca70ad4b6e8430a71d0f5e564f => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:01.881353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:01.893430 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a0ea-740d-4873-9ee2-af3f6d1c5069 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:01.966094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:01.967056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:01.967593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:01.967914 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] PATH_INFO: `/v3/projects/53ec78c2e65c4f0c86d505c409031ab6` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:01.969137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ec78c2e65c4f0c86d505c409031ab6)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:01.971072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.150460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ec78c2e65c4f0c86d505c409031ab6', 'request_id': 'req-c2a0640a-7300-46f0-bfa6-508a4dd23225'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:02.156971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ec78c2e65c4f0c86d505c409031ab6', 'request_id': 'req-c2a0640a-7300-46f0-bfa6-508a4dd23225'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:02.239982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ec78c2e65c4f0c86d505c409031ab6', 'request_id': 'req-c2a0640a-7300-46f0-bfa6-508a4dd23225'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:02.254499 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] The token cache is being invalidate because project 53ec78c2e65c4f0c86d505c409031ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:34:02.254869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:02.255671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c2a0640a-7300-46f0-bfa6-508a4dd23225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:02.261095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1252/2502] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:34:01 2022] DELETE /identity/v3/projects/53ec78c2e65c4f0c86d505c409031ab6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:34:02.262660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.263562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.263562 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.286492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:02.287736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:02.361948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1251/2503] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:02.370334 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.384688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.384688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.394028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:02.395316 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:02.404146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1253/2504] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:02.408992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.420139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db15a5-862a-4d39-b43f-7317b6bd348b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.427684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.427897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.436214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.436214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.436214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:34:02.436214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.436214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.451333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1252/2505] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:34:02.453162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.462643 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf935e6-a935-4bf9-a391-97211b2a60bf None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.468046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.468659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.468942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.469216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.469865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.471802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.563168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1254/2506] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:34:02 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:02.567604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.580148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f079-644b-46f5-98cd-7dcfa818fd30 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.641840 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.642448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.642730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.643005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.643641 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.645579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.653697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:02.721582 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1253/2507] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:34:02 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:02.725700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.737410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304221d9-d450-4c51-9711-4428047a3031 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.742516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.743121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.743401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.743698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.744346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.746021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.761841 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1255/2508] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:02.766752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.777216 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396eca9-f162-4f3a-966c-f8e366adb075 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.782679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.783322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.783607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.783915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.784567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.786159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc110b3-4257-48c9-abf8-6ed129cbe830 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.797104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1254/2509] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:34:02.801490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:02.812863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8244a-959d-44d4-bc85-dfeafe1bbf28 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:02.817623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:02.818312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.818625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.818901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] PATH_INFO: `/v3/projects/060598534767421195cef903f8543811/users/616954b7703347c6876edb1fb1df3e60/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.834829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060598534767421195cef903f8543811, user_id=616954b7703347c6876edb1fb1df3e60, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:02.837558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:02.916069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1256/2510] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:02 2022] PUT /identity/v3/projects/060598534767421195cef903f8543811/users/616954b7703347c6876edb1fb1df3e60/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:34:02.927975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:02.928266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:02.928542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:02.959659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] MFA Rules not processed for user `616954b7703347c6876edb1fb1df3e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:02.960936 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:03.032585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1255/2511] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:03.037311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.043406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:03.108930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaee505-4880-487f-8b98-11e6b92dc888 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.114168 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.114783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.115071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.115347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.116022 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.118009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.258933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1257/2512] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:34:03 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:03.263029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.274686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7199d-23a2-42de-927f-d962b02b1beb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.336573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.337466 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.337929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.338381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.339449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.342834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.440714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1256/2513] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:34:03 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:03.444926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.456017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce45a54-255d-4421-b626-8fd795e74254 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.460955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.461575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.461854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.462127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.462799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.464444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.479055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1258/2514] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:03.482852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.493095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0480c-3653-4e20-9790-fbfa2c167c7c admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.497923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.499842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.499842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.499842 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] PATH_INFO: `/v3/projects/f548f894368b427baf61f05fe39ed07c/users/14b0da2988bc4ca9a9a959d1adc4ebb3/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.517266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f548f894368b427baf61f05fe39ed07c, user_id=14b0da2988bc4ca9a9a959d1adc4ebb3, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.520202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.619470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1257/2515] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:03 2022] PUT /identity/v3/projects/f548f894368b427baf61f05fe39ed07c/users/14b0da2988bc4ca9a9a959d1adc4ebb3/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:03.623497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.634014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665148db-2842-4b63-9bb2-f82a56ced27b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.699094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.700219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.700728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.701064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.701754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.703373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d6ce-ac43-4883-b837-a9799ebb13be admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.714425 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1259/2516] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:03.719295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.730053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63167de5-cb1c-40bf-8463-d460cc165648 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.735850 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.736519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.736805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.737112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] PATH_INFO: `/v3/projects/f548f894368b427baf61f05fe39ed07c/users/14b0da2988bc4ca9a9a959d1adc4ebb3/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.739106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f548f894368b427baf61f05fe39ed07c, user_id=14b0da2988bc4ca9a9a959d1adc4ebb3, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.741829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.755272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1258/2517] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:03 2022] PUT /identity/v3/projects/f548f894368b427baf61f05fe39ed07c/users/14b0da2988bc4ca9a9a959d1adc4ebb3/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:03.766847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.767137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.767421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.797024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] MFA Rules not processed for user `14b0da2988bc4ca9a9a959d1adc4ebb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:03.798326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:03.872382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:03.877165 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1260/2518] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:03.889523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:03.891322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:03.917064 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:03.962289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b480e83-a978-4d33-8ac9-44de4ad9eee6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:03.966761 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:03.967365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:03.967672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:03.967940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:03.977434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:03.978866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3375ae0-dcb6-464c-85ad-513ec8e85e76 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:03.989148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1259/2519] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:04.305034 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:04.321776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ec3d-7428-4bd7-8624-c9dc50d25237 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:04.325989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:04.326598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:04.326897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:04.327148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:04.386610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:04.388042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:04.397107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1261/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:34:14.005251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=17 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:14.392931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=28 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:21.924586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:21.925649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:21.930642 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:21.959266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:21.970125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:22.043163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1260/2521] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:22.258002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:22.259746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:22.332960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc46f48-584f-4627-a3dc-c73eb113adab service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:22.338152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:22.338780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:22.339054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:22.339332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:22.352527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:22.354272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:22.368692 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1262/2522] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:34:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:34:27.272502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:27.281434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:27.385435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b95ad4-1239-469a-8365-f3652e5e1e17 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:27.390208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:27.391150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:27.391668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:27.392301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:27.394274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:27.507333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:27.508992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:27.520928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1261/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:32.361544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=32 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:32.755438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:32.757245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:32.785514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52677-d542-4a06-b9ab-ac49a7c84f08 service swift] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:32.791912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:32.792865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:32.793355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:32.794112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] PATH_INFO: `/v3/users/cfbdd49e2fa64b428d957d9dcf1d6846` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:32.801019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbdd49e2fa64b428d957d9dcf1d6846)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:32.808652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:33.079003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbdd49e2fa64b428d957d9dcf1d6846'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:33.173846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbdd49e2fa64b428d957d9dcf1d6846'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:33.174485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbdd49e2fa64b428d957d9dcf1d6846'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:33.175522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbdd49e2fa64b428d957d9dcf1d6846'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:33.192288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-4c18f792-cefc-4524-afce-f7b8195b97fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbdd49e2fa64b428d957d9dcf1d6846'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:33.227696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1263/2524] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:34:32 2022] DELETE /identity/v3/users/cfbdd49e2fa64b428d957d9dcf1d6846 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:33.861848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:33.878554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249c90-db94-43f6-a9ca-30aae3ea64d5 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:33.946444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:33.956264 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:33.957072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:33.957335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:33.957875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] PATH_INFO: `/v3/projects/340975b28fc242b5bf4a3364fe702ad4` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:33.960054 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=340975b28fc242b5bf4a3364fe702ad4)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:33.962710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.121430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340975b28fc242b5bf4a3364fe702ad4', 'request_id': 'req-78d8eb29-3902-4280-aca1-270153e12a5f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:34.134163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340975b28fc242b5bf4a3364fe702ad4', 'request_id': 'req-78d8eb29-3902-4280-aca1-270153e12a5f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:34.220186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340975b28fc242b5bf4a3364fe702ad4', 'request_id': 'req-78d8eb29-3902-4280-aca1-270153e12a5f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:34.239510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] The token cache is being invalidate because project 340975b28fc242b5bf4a3364fe702ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:34:34.240188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:34.240969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-78d8eb29-3902-4280-aca1-270153e12a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:34.243697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1262/2525] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:34:33 2022] DELETE /identity/v3/projects/340975b28fc242b5bf4a3364fe702ad4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:34.250112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.250868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.250868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.283091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:34.284867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:34.378609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1264/2526] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:34:34.387639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.387946 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.388410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.414433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:34.416326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:34.428570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1263/2527] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:34.435226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:34.450214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807d4fb-38bd-467b-bc89-c839000fbd4d None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:34.461520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:34.462226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.462593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.463680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.463680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:34:34.463877 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:34.465620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.484947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1265/2528] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:34:34.489353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:34.499953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c41db5-59ba-4401-b8cc-4c818911f075 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:34.505194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:34.506556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.506556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.506556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.507855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:34.510038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.649386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1264/2529] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:34:34 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:34.658438 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:34.670200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cffa50-2aec-4fab-968c-02c9d8104366 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:34.766295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:34.767322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.771209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.771209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.771209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:34.772962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.781683 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:34.913252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1266/2530] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:34:34 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:34.914520 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:34.926416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4a64-8a2d-4e4c-808e-e1011de2148b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:34.931551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:34.932190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.932471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.932748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.933391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:34.935043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.945835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1265/2531] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:34.950014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:34.961236 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb44c65-b7e0-4494-b250-7225b1e61247 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:34.970979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:34.972141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:34.972548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:34.972948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:34.974341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:34.976021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc88f-2adb-4f16-bdcd-d9d22666689a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:34.987198 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1267/2532] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:34.991580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:35.006161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba6f3-db26-49f6-8b3e-54f1b1649137 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:35.011410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:35.012109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:35.012401 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:35.012685 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] PATH_INFO: `/v3/projects/62e763646d564589a4b2e7900682a5d7/users/9de8509ca7de423cbf423bfe29dc4f0e/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:35.029592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e763646d564589a4b2e7900682a5d7, user_id=9de8509ca7de423cbf423bfe29dc4f0e, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:35.033047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:35.139084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1266/2533] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:34 2022] PUT /identity/v3/projects/62e763646d564589a4b2e7900682a5d7/users/9de8509ca7de423cbf423bfe29dc4f0e/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:35.150332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:35.150628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:35.150908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:35.198629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] MFA Rules not processed for user `9de8509ca7de423cbf423bfe29dc4f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:35.199884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:35.292477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1268/2534] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:35.410411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:35.412267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:35.497816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:35.517058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370528-d16a-4e49-8d32-cae2f16722e6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:35.525675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:35.525675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:35.525675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:35.525675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:35.532580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:35.534162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3925e4-b172-4fcd-8898-a0f6ddf2015b service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:35.543611 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1267/2535] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:43.712148 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:43.714000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:43.834332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27a093-5aa2-43b3-8280-9ded9a66011e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:43.839482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:43.840121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:43.840410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:43.840689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:43.842470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:43.852382 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:43.854952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:43.856576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695b05a-7059-4d77-98e8-395356d7fe37 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:43.867275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1269/2536] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:45.629535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=8 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:48.662941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:48.663631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:48.663631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:48.665086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1268/2537] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:34:48.671313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6673341-4d8e-47f1-a72d-bcc179194497 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:48.672358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6673341-4d8e-47f1-a72d-bcc179194497 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:48.672648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6673341-4d8e-47f1-a72d-bcc179194497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:48.743698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d6673341-4d8e-47f1-a72d-bcc179194497 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:48.745000 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d6673341-4d8e-47f1-a72d-bcc179194497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:48.777968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1270/2538] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:34:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:48.816722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:48.816722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:48.872405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8009b6-49e3-495e-87c3-6832450f386a None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:48.877610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:48.878416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:48.878821 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:48.879231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:48.892583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:48.894461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1b83-1816-44aa-b62a-da265e0480b1 service glance] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:48.907568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1269/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:49.198354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:49.198671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:49.198949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:49.200418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1271/2540] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 19:34:49.206495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:49.207002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:49.207300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:49.234157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:49.235926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:49.290938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1270/2541] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:34:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:49.304199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:49.305962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:49.320779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b74082-882a-46b8-b42b-24e9e9151eb4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:49.324339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:49.324923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:49.325252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:49.325515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:49.332944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:49.337121 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0371d8d-f6e1-46b5-8eca-009848f2553e service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:49.342200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1272/2542] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:49.742184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:49.833964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc5a7f-cd77-49ec-888e-8681b261e49c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:49.838950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:49.839757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:49.840157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:49.840344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] PATH_INFO: `/v3/users/9de8509ca7de423cbf423bfe29dc4f0e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:49.841687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de8509ca7de423cbf423bfe29dc4f0e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:49.854124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:50.219307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:50.228730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de8509ca7de423cbf423bfe29dc4f0e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:50.372579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de8509ca7de423cbf423bfe29dc4f0e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:50.373205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de8509ca7de423cbf423bfe29dc4f0e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:50.373808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de8509ca7de423cbf423bfe29dc4f0e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:50.389865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-6b1aaa1a-f20b-4a15-846d-d71fc81cbfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de8509ca7de423cbf423bfe29dc4f0e'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:50.403023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1271/2543] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:34:49 2022] DELETE /identity/v3/users/9de8509ca7de423cbf423bfe29dc4f0e => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:50.407400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:50.409912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:50.409912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:50.436649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15de84-d954-44dd-9721-1b8be825672c None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:50.436933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15de84-d954-44dd-9721-1b8be825672c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:50.437237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15de84-d954-44dd-9721-1b8be825672c None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:50.439408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1272/2544] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:34:50.445903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:50.453617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:50.453617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:50.456944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:50.458192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:50.467436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:50.468757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:50.518255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1273/2545] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:50.528067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:50.529506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:50.540174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1273/2546] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:34:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:50.551504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:50.552922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:50.599825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77539be8-cb75-41ea-a8dc-a32429d2f2f4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:50.606591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:50.614180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:50.614180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:50.614180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:50.619222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:50.620815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:50.622969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e487f3-108c-4e9e-bde0-99f2ec48c1d0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:50.627723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:50.628427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:50.628713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:50.628990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:50.631391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1274/2547] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:34:50.639780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:50.641235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb91e7-33fe-47f1-89e4-993f839e646f service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:50.653247 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1274/2548] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:34:51.195492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:51.197099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:51.226147 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c05e34-81e0-4e87-a55a-bb7f8b97ae6d service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:51.231021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:51.231712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.232033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.232354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] PATH_INFO: `/v3/projects/62e763646d564589a4b2e7900682a5d7` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.233630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e763646d564589a4b2e7900682a5d7)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:51.236521 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:51.248974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:51.463496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e763646d564589a4b2e7900682a5d7', 'request_id': 'req-1330e062-1723-40b5-af9f-8e162f9142ed'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:51.470317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e763646d564589a4b2e7900682a5d7', 'request_id': 'req-1330e062-1723-40b5-af9f-8e162f9142ed'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:51.567292 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e763646d564589a4b2e7900682a5d7', 'request_id': 'req-1330e062-1723-40b5-af9f-8e162f9142ed'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:51.578546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] The token cache is being invalidate because project 62e763646d564589a4b2e7900682a5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:34:51.578832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:51.579993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1330e062-1723-40b5-af9f-8e162f9142ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:51.581567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1275/2549] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:34:51 2022] DELETE /identity/v3/projects/62e763646d564589a4b2e7900682a5d7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:51.587587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.588113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.588241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.630880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:51.632149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:51.662738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:51.693363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1275/2550] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:51.701376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.701662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.701940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.723008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:51.724172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:51.739562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1276/2551] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:51.739715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:51.746299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b8cf9-8950-4138-8a1e-1d61697a3955 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:51.751916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:51.752521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.752805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.753082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.753735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:34:51.754040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:51.755756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:51.767971 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1276/2552] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:34:51.772514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:51.782968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f55ce6-3af4-4db7-bdfb-ea70ffb14bea None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:51.787817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:51.788441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.788756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.789035 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.789669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:51.791576 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:51.888009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1277/2553] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:34:51 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:51.893420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:51.904940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aadd8-5cc2-4bd1-addf-c729b9f62c0f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:51.986830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:51.986830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:51.986830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:51.986830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:51.987207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:51.988959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.117813 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1277/2554] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:34:51 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:52.119445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.130455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7eea7-954a-4ea6-9a34-82fddf9ff974 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.135420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.136060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.136345 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.136622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.137259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.138942 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.152223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1278/2555] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:52.161211 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.171472 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151590-41c2-484e-bde8-12592868de0a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.176585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.177173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.177454 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.177729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.178521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.180127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2606e3b-d8d2-4bf9-b3c1-8d465b305bb7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.191207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1278/2556] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:34:52.198262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.208471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227a07-f34d-4e35-a8d7-a04b2d9f2c5b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.213032 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.213675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.213956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.214232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] PATH_INFO: `/v3/projects/5b73a65d47f74376a06f747dfcba441a/users/230c6e14750b44cda7eb5cd844cf2925/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.230324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b73a65d47f74376a06f747dfcba441a, user_id=230c6e14750b44cda7eb5cd844cf2925, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.233373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.337357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1279/2557] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:52 2022] PUT /identity/v3/projects/5b73a65d47f74376a06f747dfcba441a/users/230c6e14750b44cda7eb5cd844cf2925/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:52.349329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.349678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.349960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.383240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] MFA Rules not processed for user `230c6e14750b44cda7eb5cd844cf2925`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:52.384508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:52.461787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1279/2558] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:52.466796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.530352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c796e16-719a-4b34-9307-faafffcd0df0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.535733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.536357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.536642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.536990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.537653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.539569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.655268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1280/2559] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:34:52 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:52.659398 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.672135 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e325182-64fb-4418-81f0-d5fc30b3ef74 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.699463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:52.732622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.733349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.733683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.741654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.741654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.741654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.904548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1280/2560] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:34:52 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:34:52.908355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.918998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25ae8-3d3b-4bb1-933b-b9ac8e1a0844 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.924457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.925057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.925338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.925613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.926268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.927887 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:52.939947 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1281/2561] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:52.944295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:52.954651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5688-1f04-4dfa-9cc4-294b5ce02f5a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:52.959392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:52.960030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:52.960386 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:52.960565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] PATH_INFO: `/v3/projects/49f0841683a14a7f8835d27a1126273d/users/43df07d24d33400a8d6a22e754695976/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:52.975887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f0841683a14a7f8835d27a1126273d, user_id=43df07d24d33400a8d6a22e754695976, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:52.978581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:53.087682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1281/2562] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:52 2022] PUT /identity/v3/projects/49f0841683a14a7f8835d27a1126273d/users/43df07d24d33400a8d6a22e754695976/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:53.092387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:53.103044 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083e976-c346-457a-b6cf-f930f464857b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:53.112103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:53.165517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:53.166317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:53.166567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:53.166851 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:53.167503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:53.169268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e1726-2411-497a-b221-02049c4916e3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:53.179618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1282/2563] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:34:53.184174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:53.196804 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c195b-c8aa-4931-81fc-2fbd36b3cb17 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:53.202165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:53.203536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:53.203536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:53.203536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] PATH_INFO: `/v3/projects/49f0841683a14a7f8835d27a1126273d/users/43df07d24d33400a8d6a22e754695976/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:53.206105 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f0841683a14a7f8835d27a1126273d, user_id=43df07d24d33400a8d6a22e754695976, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:53.208834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:53.219619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1282/2564] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:34:53 2022] PUT /identity/v3/projects/49f0841683a14a7f8835d27a1126273d/users/43df07d24d33400a8d6a22e754695976/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:53.230890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:53.231255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:53.231595 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:53.259785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] MFA Rules not processed for user `43df07d24d33400a8d6a22e754695976`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:53.260986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:53.336567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1283/2565] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:34:53.348123 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:53.349849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:53.413847 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d4ce8-6657-4936-8515-3c3c1282bfd9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:53.417634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:53.418239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:53.418519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:53.418991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:53.428616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:53.430046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:53.440783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1283/2566] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:53.554825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:53.566249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe0897-716d-4a81-a137-58b9dd86f36e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:53.572259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:53.573192 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:53.573573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:53.573917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:53.638515 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:53.640028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbef2-f7a0-4ee2-9654-31bffadc0f5e service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:53.656277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1284/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:59.197860 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:59.263305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931c83-148e-416a-a56e-47bef965becd service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:59.268023 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:59.268684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:59.268977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:59.269252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] PATH_INFO: `/v3/users/230c6e14750b44cda7eb5cd844cf2925` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:59.270549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=230c6e14750b44cda7eb5cd844cf2925)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:59.273210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:59.388998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:59.509050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230c6e14750b44cda7eb5cd844cf2925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:59.641614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230c6e14750b44cda7eb5cd844cf2925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:59.641869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230c6e14750b44cda7eb5cd844cf2925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:59.643252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230c6e14750b44cda7eb5cd844cf2925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:59.656124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230c6e14750b44cda7eb5cd844cf2925'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:34:59.668949 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1284/2568] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:34:59 2022] DELETE /identity/v3/users/230c6e14750b44cda7eb5cd844cf2925 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:34:59.673524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9506-9146-476e-8c99-d66215aacbbb None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:59.676112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9506-9146-476e-8c99-d66215aacbbb None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:59.676112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9506-9146-476e-8c99-d66215aacbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:59.693167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-941c9506-9146-476e-8c99-d66215aacbbb None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:34:59.694376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-941c9506-9146-476e-8c99-d66215aacbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:59.720162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:34:59.755038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1285/2569] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:34:59.764880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:34:59.766269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:34:59.849728 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dc83f-9d37-4f97-95be-df3b2124dc9f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:34:59.853307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:34:59.853908 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:59.854189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:59.859036 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:34:59.863812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:34:59.865239 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:34:59.874966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1285/2570] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:34:59.978615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:34:59.979614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:34:59.979987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:00.003928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:00.005343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:00.062615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1286/2571] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:34:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:00.076101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:00.077588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:00.163153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0708c-c251-4b91-9867-a9f7ec8ce031 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:00.170872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:00.171535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:00.172492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:00.172919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:00.183217 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:00.185018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:00.196120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1286/2572] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:35:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:35:00.353893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:00.357433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:00.379354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b735281-a855-4014-852a-395ebb1d6cc8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:00.384719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:00.385582 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:00.385790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:00.386067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] PATH_INFO: `/v3/projects/5b73a65d47f74376a06f747dfcba441a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:00.387487 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b73a65d47f74376a06f747dfcba441a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:00.389586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:00.566008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b73a65d47f74376a06f747dfcba441a', 'request_id': 'req-2e78d085-8d1c-4514-964c-8f96cbd3cb27'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:00.619112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b73a65d47f74376a06f747dfcba441a', 'request_id': 'req-2e78d085-8d1c-4514-964c-8f96cbd3cb27'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:00.708530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b73a65d47f74376a06f747dfcba441a', 'request_id': 'req-2e78d085-8d1c-4514-964c-8f96cbd3cb27'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:00.737593 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] The token cache is being invalidate because project 5b73a65d47f74376a06f747dfcba441a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:00.738167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:00.738935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:00.745261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1287/2573] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:00 2022] DELETE /identity/v3/projects/5b73a65d47f74376a06f747dfcba441a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:00.749425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:00.775326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf5996-58cd-400e-a2c8-ff98999e1fb1 service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:00.799600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:00.845349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:00.845349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:00.883678 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:00.884362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:00.884716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:00.885150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] PATH_INFO: `/v3/users/43df07d24d33400a8d6a22e754695976` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:00.886494 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43df07d24d33400a8d6a22e754695976)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:00.888616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:00.924720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d085-8d1c-4514-964c-8f96cbd3cb27 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:00.935286 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:00.936282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:00.937284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:00.937935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] PATH_INFO: `/v3/users/616954b7703347c6876edb1fb1df3e60` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:00.940227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616954b7703347c6876edb1fb1df3e60)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:00.942330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:00.944569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:01.082300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43df07d24d33400a8d6a22e754695976'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.184289 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43df07d24d33400a8d6a22e754695976'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.185167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43df07d24d33400a8d6a22e754695976'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.185994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43df07d24d33400a8d6a22e754695976'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.203689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-51f0268c-d7f7-4548-a328-ad3d9e74fb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43df07d24d33400a8d6a22e754695976'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.222782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1287/2574] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:00 2022] DELETE /identity/v3/users/43df07d24d33400a8d6a22e754695976 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:01.242434 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616954b7703347c6876edb1fb1df3e60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.327098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616954b7703347c6876edb1fb1df3e60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.328003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616954b7703347c6876edb1fb1df3e60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.328829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616954b7703347c6876edb1fb1df3e60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.350158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616954b7703347c6876edb1fb1df3e60'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:01.375824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1288/2575] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:00 2022] DELETE /identity/v3/users/616954b7703347c6876edb1fb1df3e60 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:01.381352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:01.381763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:01.382165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:01.406577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:01.408835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:01.480068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1288/2576] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:01.490153 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:01.492151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:01.583167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542ccc-b04b-4ccd-a28b-435e56f5fa99 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:01.588054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:01.588684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:01.589131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:01.589337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:01.620209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:01.622017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad0ca8-1753-426b-a019-a4732abfcd77 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:01.650008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1289/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:01.859772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:01.862346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:01.895563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128bdf5-3cb1-473a-89de-6b26a71be106 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:01.901461 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:01.902181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:01.902601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:01.903546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] PATH_INFO: `/v3/projects/49f0841683a14a7f8835d27a1126273d` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:01.917972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f0841683a14a7f8835d27a1126273d)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:01.920357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:02.043822 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:02.063917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f0841683a14a7f8835d27a1126273d', 'request_id': 'req-c2e1e846-06e6-4f1f-98f6-a64879031076'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.083324 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f0841683a14a7f8835d27a1126273d', 'request_id': 'req-c2e1e846-06e6-4f1f-98f6-a64879031076'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.235951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f0841683a14a7f8835d27a1126273d', 'request_id': 'req-c2e1e846-06e6-4f1f-98f6-a64879031076'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.257805 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] The token cache is being invalidate because project 49f0841683a14a7f8835d27a1126273d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:02.258163 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.258901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c2e1e846-06e6-4f1f-98f6-a64879031076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.261331 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1289/2578] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:01 2022] DELETE /identity/v3/projects/49f0841683a14a7f8835d27a1126273d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:02.266527 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.267751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.268298 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.295830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:02.297771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:02.378489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1290/2579] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:02.387468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.387903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.388221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.424891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:02.426093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:02.434802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1290/2580] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:02.439804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.449972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a3931-9239-4e7b-9777-378d47a18f8c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:02.454767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:02.457028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.457028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.457028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.457028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:02.457028 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:02.459904 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:02.475696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1291/2581] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:02.478392 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.490438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df7c4-cc8b-4bf3-ba23-e1509561211f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:02.495948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:02.496306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.496586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.496903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.497487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:02.499587 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:02.532008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.532305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:02.560625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bbf56-b794-421d-82ca-d31356b8a18d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:02.572580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:02.579939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.579939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.579939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] PATH_INFO: `/v3/projects/060598534767421195cef903f8543811` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.579939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] RBAC: Authorizing `identity:delete_project(project_id=060598534767421195cef903f8543811)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:02.581401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:02.598737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:02.629372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1291/2582] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:02 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:02.632712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.720610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82475e71-2a8d-4632-aea0-a73d55354cbc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:02.726565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:02.727192 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.727477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.727851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.728500 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:02.730446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:02.788713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060598534767421195cef903f8543811', 'request_id': 'req-f3e01b7b-08f4-458c-b38f-015ee982052c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.796639 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060598534767421195cef903f8543811', 'request_id': 'req-f3e01b7b-08f4-458c-b38f-015ee982052c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.854000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1292/2583] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:02 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:02.858259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.877546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841433a-9ac0-402a-99ea-c7cd8cea8ec3 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:02.882664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060598534767421195cef903f8543811', 'request_id': 'req-f3e01b7b-08f4-458c-b38f-015ee982052c'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.895705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] The token cache is being invalidate because project 060598534767421195cef903f8543811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:02.896099 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.896734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:02.898347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1292/2584] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:02 2022] DELETE /identity/v3/projects/060598534767421195cef903f8543811 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:02.902186 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:02.903759 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:02.985448 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:02.986107 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:02.986406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:02.986688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:02.987344 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:02.991621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:03.008164 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1293/2585] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:03.011878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:03.013779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:03.018624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e01b7b-08f4-458c-b38f-015ee982052c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:03.024661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:03.025955 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:03.026318 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:03.026610 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] PATH_INFO: `/v3/users/14b0da2988bc4ca9a9a959d1adc4ebb3` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:03.027863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b0da2988bc4ca9a9a959d1adc4ebb3)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:03.030056 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:03.038653 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767916-f15a-4c0d-af66-4bf21de68a00 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:03.044342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:03.045244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:03.045718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:03.046182 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:03.047329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:03.049024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:03.068700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1294/2586] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:03.069923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:03.086610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:03.090733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9d331-e86e-4d78-9d9f-43a4dbdb53ec admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:03.095955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:03.096619 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:03.096902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:03.097181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] PATH_INFO: `/v3/projects/2a0766394a544fa0b025d132a3537caa/users/29161db8c5884334a379840d5169fecb/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:03.121968 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0766394a544fa0b025d132a3537caa, user_id=29161db8c5884334a379840d5169fecb, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:03.126658 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f2842-25e2-41dd-9551-a2b19e865d3b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:03.276376 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1295/2587] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:03 2022] PUT /identity/v3/projects/2a0766394a544fa0b025d132a3537caa/users/29161db8c5884334a379840d5169fecb/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:03.287011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:03.287306 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:03.287584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:03.320079 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] MFA Rules not processed for user `29161db8c5884334a379840d5169fecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:03.321330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:03.345048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b0da2988bc4ca9a9a959d1adc4ebb3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:03.433444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b0da2988bc4ca9a9a959d1adc4ebb3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:03.434072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b0da2988bc4ca9a9a959d1adc4ebb3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:03.434660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b0da2988bc4ca9a9a959d1adc4ebb3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:03.448171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1296/2588] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:03.454299 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:03.469394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b0da2988bc4ca9a9a959d1adc4ebb3'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:03.497549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1293/2589] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:02 2022] DELETE /identity/v3/users/14b0da2988bc4ca9a9a959d1adc4ebb3 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:03.568928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bf774-9e91-49c9-a8c9-9c0c082e7933 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:03.659373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:03.660017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:03.660302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:03.660655 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:03.661276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:03.663173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:03.803235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1297/2590] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:03 2022] POST /identity/v3/projects => generated 389 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:03.804355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:03.806493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:03.825719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d717e-1e5c-4048-b8b0-8760a9734b3c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:03.837423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:04.082717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.083328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.083621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.083982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.084632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.086680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.205853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1294/2591] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:03 2022] POST /identity/v3/users => generated 301 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:04.210332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.222651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:04.222651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74a78-0773-42d0-ba11-e260b6b29c2e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.227276 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.228861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.228861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.228861 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.229764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.231823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.251108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1298/2592] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:04.256094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.269263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.271542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36478390-e7ae-481c-b6a1-0dd42b8f8aca admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.272374 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.294778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.295467 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.295827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.296059 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] PATH_INFO: `/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98/users/4aa8241d47c74665bdfd78375f3c37f2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.313682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e9f76-f736-4a06-8d91-c4b7047b37fd admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.315535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a77edb052e43c6b49cc2f6fcac9b98, user_id=4aa8241d47c74665bdfd78375f3c37f2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.318274 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.320370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.321254 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.321600 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.322574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] PATH_INFO: `/v3/projects/f548f894368b427baf61f05fe39ed07c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.324091 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f548f894368b427baf61f05fe39ed07c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.326346 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.402865 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1295/2593] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:04 2022] PUT /identity/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98/users/4aa8241d47c74665bdfd78375f3c37f2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:04.406952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.496258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ac1f1-2243-4040-b596-a492e5d70a97 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.513266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f548f894368b427baf61f05fe39ed07c', 'request_id': 'req-444bae9d-e98e-45f3-9404-205b5531e827'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:04.526624 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f548f894368b427baf61f05fe39ed07c', 'request_id': 'req-444bae9d-e98e-45f3-9404-205b5531e827'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:04.606006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.607220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.608090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.608588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.609672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.612870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.613024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f548f894368b427baf61f05fe39ed07c', 'request_id': 'req-444bae9d-e98e-45f3-9404-205b5531e827'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:04.641739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] The token cache is being invalidate because project f548f894368b427baf61f05fe39ed07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:04.642103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:04.647525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1296/2594] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:04.647757 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-444bae9d-e98e-45f3-9404-205b5531e827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:04.653195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1299/2595] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:04 2022] DELETE /identity/v3/projects/f548f894368b427baf61f05fe39ed07c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:04.653414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.658681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.659321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6de7-25cb-46e3-87e9-df88919573e3 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.659613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6de7-25cb-46e3-87e9-df88919573e3 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.659918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6de7-25cb-46e3-87e9-df88919573e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.683295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332484a6-bea4-425d-b8ea-ed2b4c0003bc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.696861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.698968 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.699471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.700247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] PATH_INFO: `/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98/users/4aa8241d47c74665bdfd78375f3c37f2/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.702846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-b6db6de7-25cb-46e3-87e9-df88919573e3 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:04.704129 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b6db6de7-25cb-46e3-87e9-df88919573e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.705648 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a77edb052e43c6b49cc2f6fcac9b98, user_id=4aa8241d47c74665bdfd78375f3c37f2, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.723220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1300/2596] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:04.727682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.727682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.727682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.731714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309d128-d89a-4972-aa3d-6689ab0196bf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.764053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:04.768160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1297/2597] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:04 2022] PUT /identity/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98/users/4aa8241d47c74665bdfd78375f3c37f2/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:04.768382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.777296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.777737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.778190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.797219 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:04.825048 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] MFA Rules not processed for user `4aa8241d47c74665bdfd78375f3c37f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:04.826869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.886147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1301/2598] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:04.890957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.902267 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4378-497e-468a-be72-60d8d7ba22bb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.912824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.913886 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.914377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.914855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.916046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:04.916597 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.918657 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:04.932744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1302/2599] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:04.937258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1298/2600] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:04.942357 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.955071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:04.955071 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:04.959316 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91284730-f54f-413c-9a85-8a1bbbe4d2bc admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:04.972219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:04.973230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:04.973696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:04.974393 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:04.975789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:04.979463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.088234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ce939-ce32-4557-b01c-b13dd090dd40 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.092825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.093452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.093734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.094046 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.104979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.106700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.128790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1303/2601] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:04 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:05.129819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1299/2602] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:05.136961 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.230367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:05.305292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0213888-03a4-4229-817a-08508d3c873e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.312345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.313144 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.313689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.314009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.314610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.316889 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.375687 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.391279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce00e2-05e3-4453-b505-88781af9b0ae service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.401439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.402509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.404733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.404733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] PATH_INFO: `/v3/users/29161db8c5884334a379840d5169fecb` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.411072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29161db8c5884334a379840d5169fecb)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.414757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.429831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1304/2603] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:05 2022] POST /identity/v3/users => generated 294 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:05.435246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.458803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268590f-c042-4fda-8e9f-be98d1f75a08 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.482736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.483523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.483937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.484318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.485108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.487511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.503414 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1305/2604] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:05.507638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.519364 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd28e-f8db-4da1-a9a6-a78251db8f31 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.524798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.525662 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.526050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.526433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.527191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.528981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.554233 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1306/2605] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:05.558428 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.575038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3794-0723-49de-a004-c537947d2b7f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:05.580990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:05.581677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.582125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.582403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] PATH_INFO: `/v3/projects/51f0374754fa4dc8b0b6c634adb9bc0c/users/e09516bee79e4c03b9dfe42b2041fb6c/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.599910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f0374754fa4dc8b0b6c634adb9bc0c, user_id=e09516bee79e4c03b9dfe42b2041fb6c, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:05.602630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01c8e-6340-4dab-a067-0970d06cb5bb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:05.645536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29161db8c5884334a379840d5169fecb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:05.715714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1307/2606] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:05 2022] PUT /identity/v3/projects/51f0374754fa4dc8b0b6c634adb9bc0c/users/e09516bee79e4c03b9dfe42b2041fb6c/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:05.733623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29161db8c5884334a379840d5169fecb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:05.733838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.734151 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.735930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29161db8c5884334a379840d5169fecb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:05.736143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.736255 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29161db8c5884334a379840d5169fecb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:05.755045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-255782b0-6837-44d1-8521-ed616cb46e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29161db8c5884334a379840d5169fecb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:05.778664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:05.795544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] MFA Rules not processed for user `e09516bee79e4c03b9dfe42b2041fb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:05.796853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:05.824793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1300/2607] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:05 2022] DELETE /identity/v3/users/29161db8c5884334a379840d5169fecb => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:05.831254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:05.831696 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:05.832214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:05.869164 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:05.870580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:05.909420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1308/2608] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:05.910941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:05.912680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:06.026518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02902-c629-4f37-88cd-959394f8dd49 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.041214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1301/2609] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:06.046925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.047455 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.047626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.047926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.048158 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.048858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.049218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:06.052477 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.073687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:06.145470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd34a3-7e20-4fe0-9007-2bd66d73343b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.149664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.150373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.150725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.151073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.161489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.166006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.175715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1302/2610] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:06.199415 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1309/2611] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:05 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:06.207466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.224162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71283-375a-462c-b96c-5b9748141d96 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.303428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.304069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.304722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.308117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.310447 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.315183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.429259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1303/2612] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:06 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:06.433511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.450888 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b77fd-732f-4540-a28c-e5fb1801b1f6 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.461899 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.463032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.463910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.464350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.465711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.469372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.501877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1310/2613] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:06.502642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.531199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb0602-7173-40ee-aa84-cc00720c35f9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.543407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.544954 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.545490 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.545948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] PATH_INFO: `/v3/projects/4d22c413af4940b6995ba4d6be0573f2/users/5926318387804e5c993806cece9171c0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.562741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d22c413af4940b6995ba4d6be0573f2, user_id=5926318387804e5c993806cece9171c0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.566414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.570008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:06.656312 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1304/2614] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:06 2022] PUT /identity/v3/projects/4d22c413af4940b6995ba4d6be0573f2/users/5926318387804e5c993806cece9171c0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:06.657626 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.671726 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f60d4-5b61-4769-91e8-31d166a67b39 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.746424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.748604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:06.819308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.820771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.821539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.822051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.822975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.825569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.839639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1311/2615] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:06.844273 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:06.854637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929d2e0-c734-4eda-a558-e8658b04234f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.864575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:06.866567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.866990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.867432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] PATH_INFO: `/v3/projects/4d22c413af4940b6995ba4d6be0573f2/users/5926318387804e5c993806cece9171c0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.869835 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d22c413af4940b6995ba4d6be0573f2, user_id=5926318387804e5c993806cece9171c0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:06.873028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496550-7ac3-4eb1-9ec5-eb2f8b3b795e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:06.898212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1312/2616] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:06 2022] PUT /identity/v3/projects/4d22c413af4940b6995ba4d6be0573f2/users/5926318387804e5c993806cece9171c0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:06.905741 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:06.906820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:06.907212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:06.942306 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f309518-c4e3-4b1a-9163-60762a29feec admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:06.960730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] MFA Rules not processed for user `5926318387804e5c993806cece9171c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:06.961540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:07.068990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1313/2617] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:07.088609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:07.089146 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:07.089496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:07.089730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] PATH_INFO: `/v3/projects/2a0766394a544fa0b025d132a3537caa` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:07.091374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0766394a544fa0b025d132a3537caa)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:07.093672 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:07.107959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:07.110422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:07.243832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:07.270929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e6e6b-480b-4a03-ae41-91e165815e4f None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:07.270929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:07.270929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:07.270929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:07.271377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:07.271377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:07.271921 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:07.282445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1314/2618] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:07.329257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:07.353940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bf5b3-1554-4bd2-aa7e-243009dc5f0f service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:07.367827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:07.369229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:07.370465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:07.370783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] PATH_INFO: `/v3/users/e09516bee79e4c03b9dfe42b2041fb6c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:07.375698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09516bee79e4c03b9dfe42b2041fb6c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:07.377675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:07.465579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0766394a544fa0b025d132a3537caa', 'request_id': 'req-c90a8160-b9a4-4d12-880a-2873f5f318d6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.476881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0766394a544fa0b025d132a3537caa', 'request_id': 'req-c90a8160-b9a4-4d12-880a-2873f5f318d6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.555951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0766394a544fa0b025d132a3537caa', 'request_id': 'req-c90a8160-b9a4-4d12-880a-2873f5f318d6'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.570697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] The token cache is being invalidate because project 2a0766394a544fa0b025d132a3537caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:07.571296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.572202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.580833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1305/2619] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:06 2022] DELETE /identity/v3/projects/2a0766394a544fa0b025d132a3537caa => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:07.584492 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:07.587731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:07.650957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09516bee79e4c03b9dfe42b2041fb6c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.672063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:07.722827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09516bee79e4c03b9dfe42b2041fb6c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.724246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09516bee79e4c03b9dfe42b2041fb6c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.725612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09516bee79e4c03b9dfe42b2041fb6c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.751613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a8160-b9a4-4d12-880a-2873f5f318d6 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:07.759118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:07.759802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:07.760095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:07.760373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] PATH_INFO: `/v3/users/4aa8241d47c74665bdfd78375f3c37f2` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:07.761649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa8241d47c74665bdfd78375f3c37f2)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:07.763766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:07.777473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-2953ec84-1f8c-4e26-88b5-213648a2ef66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09516bee79e4c03b9dfe42b2041fb6c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:07.804358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1315/2620] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:07 2022] DELETE /identity/v3/users/e09516bee79e4c03b9dfe42b2041fb6c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:07.810009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:07.810567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:07.810995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:07.835257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:07.837543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:07.932722 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1316/2621] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:07.943700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:07.945564 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:08.037053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04400eb-5b38-42c2-92a7-1e6166c4808e None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:08.042322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:08.043109 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:08.043248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:08.043700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:08.054695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:08.056090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:08.057543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa8241d47c74665bdfd78375f3c37f2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.068563 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1317/2622] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:08.210464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa8241d47c74665bdfd78375f3c37f2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.211201 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa8241d47c74665bdfd78375f3c37f2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.212040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa8241d47c74665bdfd78375f3c37f2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.220611 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa8241d47c74665bdfd78375f3c37f2'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.239359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1306/2623] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:07 2022] DELETE /identity/v3/users/4aa8241d47c74665bdfd78375f3c37f2 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:08.669033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:08.672440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:08.689681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ed094-360f-495f-b2f3-8dc2f02a5bb0 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:08.698277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:08.770510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:08.773751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:08.788006 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:08.788706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:08.788991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:08.789275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] PATH_INFO: `/v3/projects/51f0374754fa4dc8b0b6c634adb9bc0c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:08.790763 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f0374754fa4dc8b0b6c634adb9bc0c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:08.793404 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:08.876916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27cc9-cf03-4cad-bda3-11e73013ffa5 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:08.881586 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:08.882287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:08.882567 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:08.882840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] PATH_INFO: `/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:08.884008 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a77edb052e43c6b49cc2f6fcac9b98)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:08.885991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:08.930136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f0374754fa4dc8b0b6c634adb9bc0c', 'request_id': 'req-f2a9e025-1b4b-4c57-84ac-0704abd2d105'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:08.938018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f0374754fa4dc8b0b6c634adb9bc0c', 'request_id': 'req-f2a9e025-1b4b-4c57-84ac-0704abd2d105'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.016779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f0374754fa4dc8b0b6c634adb9bc0c', 'request_id': 'req-f2a9e025-1b4b-4c57-84ac-0704abd2d105'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.034727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a77edb052e43c6b49cc2f6fcac9b98', 'request_id': 'req-f4662184-5108-4617-931a-f05ad8324558'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.035444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] The token cache is being invalidate because project 51f0374754fa4dc8b0b6c634adb9bc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:09.036005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.038155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.041100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1318/2624] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:08 2022] DELETE /identity/v3/projects/51f0374754fa4dc8b0b6c634adb9bc0c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:09.047263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:09.048647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a77edb052e43c6b49cc2f6fcac9b98', 'request_id': 'req-f4662184-5108-4617-931a-f05ad8324558'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.049945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:09.053512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:09.128791 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a77edb052e43c6b49cc2f6fcac9b98', 'request_id': 'req-f4662184-5108-4617-931a-f05ad8324558'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.149952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e025-1b4b-4c57-84ac-0704abd2d105 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:09.155231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] The token cache is being invalidate because project 07a77edb052e43c6b49cc2f6fcac9b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:09.155886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.157057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-f4662184-5108-4617-931a-f05ad8324558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.158935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1307/2625] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:08 2022] DELETE /identity/v3/projects/07a77edb052e43c6b49cc2f6fcac9b98 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:09.167353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133ab87-4992-486a-9df1-0536a1c93e5b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.167960 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133ab87-4992-486a-9df1-0536a1c93e5b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.170222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133ab87-4992-486a-9df1-0536a1c93e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.200786 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-5133ab87-4992-486a-9df1-0536a1c93e5b None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:09.203049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5133ab87-4992-486a-9df1-0536a1c93e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:09.244307 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:09.244955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.245237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.245511 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] PATH_INFO: `/v3/users/5926318387804e5c993806cece9171c0` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.246776 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5926318387804e5c993806cece9171c0)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:09.248903 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:09.297713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1308/2626] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:09.308190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.308534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.309054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.334276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:09.339829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:09.351510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1309/2627] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:09.353585 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:09.367901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100439e-6d90-4679-bded-c736226e4a8a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:09.373790 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:09.374629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.374907 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.375197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.375909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:09.376213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:09.378358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:09.392727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1310/2628] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:35:09.398740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:09.419095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544d18-e369-428c-8fe8-619a6159f4d0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:09.425778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:09.426393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.426824 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.428001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.428994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:09.432317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:09.487730 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:09.629673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1311/2629] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:09 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:09.634463 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:09.649488 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5926318387804e5c993806cece9171c0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.716393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0688bc5-5b12-4a53-9c22-bd572ec309c7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:09.721781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:09.728657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.728657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.728657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.728657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:09.728657 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:09.755938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5926318387804e5c993806cece9171c0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.758235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5926318387804e5c993806cece9171c0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.758235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5926318387804e5c993806cece9171c0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.779704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5926318387804e5c993806cece9171c0'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:09.795085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1319/2630] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:09 2022] DELETE /identity/v3/users/5926318387804e5c993806cece9171c0 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:09.875967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1312/2631] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:09 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:09.881474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:09.881474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:09.982893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabcc2-a0a8-4d6c-852d-c8e89e0e8a5e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:09.988645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:09.989260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:09.989589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:09.989875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:09.990528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:09.992170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.002580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1320/2632] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:10.010876 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.017078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78e70-1b42-41cd-bc55-186f8a9f4995 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.021690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.022372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.022653 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.022927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.023563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.025182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967df14-c873-448f-98e8-f6bc65975773 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.036114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1313/2633] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:10.041580 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.056349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bb8e2-6d67-4530-814f-e0b66092adc8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.061422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.062098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.062389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.062670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] PATH_INFO: `/v3/projects/1772e196a92e49a78b88a183f5187773/users/b5548432b584426390aa0d60abacc8cb/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.081883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1772e196a92e49a78b88a183f5187773, user_id=b5548432b584426390aa0d60abacc8cb, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.084200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.173181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1321/2634] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:10 2022] PUT /identity/v3/projects/1772e196a92e49a78b88a183f5187773/users/b5548432b584426390aa0d60abacc8cb/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:10.187691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.189067 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.189353 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.224160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] MFA Rules not processed for user `b5548432b584426390aa0d60abacc8cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:10.225397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:10.237983 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:10.282034 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.285047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:10.312680 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1314/2635] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:10.317663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.358746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:10.384183 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4c18-aea9-4815-8126-e893b634772a admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.389787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.390567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.390953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.391327 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] PATH_INFO: `/v3/projects/4d22c413af4940b6995ba4d6be0573f2` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.393199 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d22c413af4940b6995ba4d6be0573f2)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.395265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.410922 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e28d49-0ed0-465e-99c0-8e19069ec7d9 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.415976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.416587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.416881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.417144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.417777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.419930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.528775 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1315/2636] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:10.535917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.582586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d22c413af4940b6995ba4d6be0573f2', 'request_id': 'req-73d9e933-ac95-471b-91ca-60dbdd1233f5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:10.589008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d22c413af4940b6995ba4d6be0573f2', 'request_id': 'req-73d9e933-ac95-471b-91ca-60dbdd1233f5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:10.600523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b46e-5f1f-470c-96f5-e863beb7edfc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.657969 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.658498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.658815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.659094 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.659770 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.661686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.696167 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d22c413af4940b6995ba4d6be0573f2', 'request_id': 'req-73d9e933-ac95-471b-91ca-60dbdd1233f5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:10.708397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] The token cache is being invalidate because project 4d22c413af4940b6995ba4d6be0573f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:10.708690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:10.709359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-73d9e933-ac95-471b-91ca-60dbdd1233f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:10.710872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1322/2637] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:10 2022] DELETE /identity/v3/projects/4d22c413af4940b6995ba4d6be0573f2 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:10.715753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a4dd-b74e-47b1-a738-fcc53d763ad7 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.716050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a4dd-b74e-47b1-a738-fcc53d763ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.716334 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a4dd-b74e-47b1-a738-fcc53d763ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.746604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6c43a4dd-b74e-47b1-a738-fcc53d763ad7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:10.747869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6c43a4dd-b74e-47b1-a738-fcc53d763ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:10.756736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1323/2638] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:10.765708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.766244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.768173 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.774346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1316/2639] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:10.778580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.780399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:10.792002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:10.797044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:10.797776 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129b85b-abd2-4526-a9d5-10915554a87f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.802689 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1324/2640] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:10.807607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.808384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.808805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.809134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.809308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.810162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.812220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.821549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf9fdb-ae94-4f90-8fbf-20afc8a3a9db None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.826439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1317/2641] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:10.828168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.828902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.829288 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.829837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.830867 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:10.831321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.832805 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.833011 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.844982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eec4f-a069-4de5-8e73-37c8952532cb admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.848914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1325/2642] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:10.850483 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.851253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.851530 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.851834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] PATH_INFO: `/v3/projects/d7df25627381412f962a6c2e3ab4875b/users/2eb91e50f22d407d9ad61ab1c6a3ebb7/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.853210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.866851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cd8b-f780-4978-b1da-46df6cafe689 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:10.868743 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7df25627381412f962a6c2e3ab4875b, user_id=2eb91e50f22d407d9ad61ab1c6a3ebb7, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.872589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:10.875966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:10.876098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.876546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:10.876546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:10.876546 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:10.876929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:10.949768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1318/2643] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:10 2022] PUT /identity/v3/projects/d7df25627381412f962a6c2e3ab4875b/users/2eb91e50f22d407d9ad61ab1c6a3ebb7/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:10.953859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:10.978367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:10.983391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1326/2644] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:10.992846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.050256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e90905-a871-43bb-a254-d0defe1bf8a3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.084995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0a8-14d0-4ed3-92aa-c72d9c5e2027 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.091137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.091756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.092143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.092561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.093208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.095255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.159940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.160559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.160843 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.161114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.161764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.163352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.174172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1319/2645] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:11.178471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.188733 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3329e82-d84c-48cf-aba3-afba3086d457 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.194087 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.194734 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.195016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.195321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] PATH_INFO: `/v3/projects/d7df25627381412f962a6c2e3ab4875b/users/2eb91e50f22d407d9ad61ab1c6a3ebb7/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.197237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7df25627381412f962a6c2e3ab4875b, user_id=2eb91e50f22d407d9ad61ab1c6a3ebb7, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.199867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962c1c1-7377-4924-8db8-dfb3d379d1de admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.209857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1320/2646] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:11 2022] PUT /identity/v3/projects/d7df25627381412f962a6c2e3ab4875b/users/2eb91e50f22d407d9ad61ab1c6a3ebb7/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:11.220375 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.220771 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.221055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.226460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1327/2647] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:10 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:11.233912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.245073 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01fedb-7df2-4c16-90ce-5b5720703204 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.249059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:11.259589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] MFA Rules not processed for user `2eb91e50f22d407d9ad61ab1c6a3ebb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:11.260936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:11.308068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.308681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.308973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.309244 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.310048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.311634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.322668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1328/2648] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:11.327071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.337195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15e03f-5e21-4ea8-a75d-5d7f5ac03ac0 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.341980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.342575 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.342863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.343138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.343823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.345403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.356280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1321/2649] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:11.362959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1329/2650] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:11.368095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.374838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.376613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:11.380412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80058e-128d-4d3d-82e4-d59d2e1c95a1 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.385391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:11.394152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.394152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.394152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.394152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] PATH_INFO: `/v3/projects/227df7dd1a5a45c7bab4e8b7cfbefba6/users/3e95318c7fe44d9db264c83aae2e398d/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.410489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=227df7dd1a5a45c7bab4e8b7cfbefba6, user_id=3e95318c7fe44d9db264c83aae2e398d, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.413250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289cd5d-7aaa-4412-a464-96ab7d61b4ee admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.443536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9356a-80ac-479b-b9f0-541eeb0eb7a9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.447853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.449555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.449555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.449555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.450781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:11.494539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1322/2651] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:11 2022] PUT /identity/v3/projects/227df7dd1a5a45c7bab4e8b7cfbefba6/users/3e95318c7fe44d9db264c83aae2e398d/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:11.504704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.504998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.505272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.519896 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.521345 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.530630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1330/2652] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:11.537355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] MFA Rules not processed for user `3e95318c7fe44d9db264c83aae2e398d`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:11.539060 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:11.619712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1323/2653] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:11.622198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.641194 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cfe35-1b5b-4b32-80ee-5bcb362884df service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.700670 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:11.728427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.729052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.729338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.729671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.730318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.732397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:11.877984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1331/2654] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:11 2022] POST /identity/v3/projects => generated 381 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:11.882084 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:11.893751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8084-1c64-4346-ba22-57a2b5937db4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:11.958993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:11.958993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:11.958993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:11.959424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:11.959933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:11.961842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.042572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1324/2655] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:11 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:12.052017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.057282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185935c-daf2-4a7e-920b-124b11637154 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.062222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.062816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.063101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.063373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.064803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.066388 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.077385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1332/2656] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:12.084409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.095779 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac459ad-8605-40dd-a2b1-3e108d6f1d06 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.101444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.102901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.109302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.109302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] PATH_INFO: `/v3/projects/b33157dbeaf84869adf98a90eb87d8a4/users/631288ca91f349ddac17ba5e34081df4/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.112237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:12.130179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33157dbeaf84869adf98a90eb87d8a4, user_id=631288ca91f349ddac17ba5e34081df4, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.132957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.224638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1325/2657] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:12 2022] PUT /identity/v3/projects/b33157dbeaf84869adf98a90eb87d8a4/users/631288ca91f349ddac17ba5e34081df4/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:12.228900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.242015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46738142-3475-4cf4-a6ef-3bc137981a96 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.335539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.335801 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.336089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.336367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.337015 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.338687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900471b-666e-4d17-989b-dc5b0e550193 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.349992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1333/2658] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:12.355541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.366649 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ed0a1-a6f5-4996-a8a6-5406d4a7d1c3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.371512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.372287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.372569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.372847 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] PATH_INFO: `/v3/projects/b33157dbeaf84869adf98a90eb87d8a4/users/631288ca91f349ddac17ba5e34081df4/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.375177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33157dbeaf84869adf98a90eb87d8a4, user_id=631288ca91f349ddac17ba5e34081df4, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.382565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.393124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1326/2659] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:12 2022] PUT /identity/v3/projects/b33157dbeaf84869adf98a90eb87d8a4/users/631288ca91f349ddac17ba5e34081df4/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:12.404826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.405455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.406125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.449240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] MFA Rules not processed for user `631288ca91f349ddac17ba5e34081df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:12.450486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:12.543125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1334/2660] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:12.555778 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.615237 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9652-68aa-44ef-9d5f-6c74a74278a1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.619019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.619628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.619948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.620250 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.630294 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.631753 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.641309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1327/2661] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:12.803752 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:12.813314 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206a712-8c3e-4b65-bbe0-6d444652efe2 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:12.817241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:12.817863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:12.818150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:12.818427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:12.863012 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:12.878852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:12.880353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554ee1-282d-4be2-b607-29c206f403a4 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:12.890988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1335/2662] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:13.454308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:13.552045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cbcd1-4752-4373-a0c3-268e3de9f988 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:13.556813 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:13.557387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:13.557666 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:13.557937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] PATH_INFO: `/v3/users/3e95318c7fe44d9db264c83aae2e398d` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:13.559049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e95318c7fe44d9db264c83aae2e398d)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:13.561224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:13.770097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e95318c7fe44d9db264c83aae2e398d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:13.862962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e95318c7fe44d9db264c83aae2e398d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:13.863518 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e95318c7fe44d9db264c83aae2e398d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:13.864305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e95318c7fe44d9db264c83aae2e398d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:13.871935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e95318c7fe44d9db264c83aae2e398d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:13.880631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:13.890630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1328/2663] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:13 2022] DELETE /identity/v3/users/3e95318c7fe44d9db264c83aae2e398d => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:13.895340 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:13.895837 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:13.895950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:13.915916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:13.918467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:13.981904 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1336/2664] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:13.992629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:13.994042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:14.058891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355c218-e3d3-4f38-ab12-86d9c25b9457 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:14.066705 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:14.067324 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:14.067609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:14.067912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:14.078366 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:14.079861 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:14.090395 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1329/2665] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:14.605838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:14.607427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:14.620731 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252912c4-5eee-497d-be1c-b0217fd05bb6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:14.625691 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:14.626336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:14.626643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:14.626923 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] PATH_INFO: `/v3/projects/227df7dd1a5a45c7bab4e8b7cfbefba6` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:14.628295 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=227df7dd1a5a45c7bab4e8b7cfbefba6)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:14.630280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:14.822161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:14.823793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:14.837722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b32d6-d11c-44d8-89ad-8b5413be39d6 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:14.843344 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:14.843994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:14.844275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:14.844548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] PATH_INFO: `/v3/users/b5548432b584426390aa0d60abacc8cb` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:14.845661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5548432b584426390aa0d60abacc8cb)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:14.848277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:15.634865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227df7dd1a5a45c7bab4e8b7cfbefba6', 'request_id': 'req-e5e81972-767e-43fe-9584-bbd603582abe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.642440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227df7dd1a5a45c7bab4e8b7cfbefba6', 'request_id': 'req-e5e81972-767e-43fe-9584-bbd603582abe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.762171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227df7dd1a5a45c7bab4e8b7cfbefba6', 'request_id': 'req-e5e81972-767e-43fe-9584-bbd603582abe'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.781180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] The token cache is being invalidate because project 227df7dd1a5a45c7bab4e8b7cfbefba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:15.781967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.782998 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.785322 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5548432b584426390aa0d60abacc8cb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.786017 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1337/2666] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:14 2022] DELETE /identity/v3/projects/227df7dd1a5a45c7bab4e8b7cfbefba6 => generated 0 bytes in 1180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:15.790021 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:15.791539 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:15.871422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e81972-767e-43fe-9584-bbd603582abe admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:15.877045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:15.877714 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:15.878170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:15.878609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] PATH_INFO: `/v3/users/631288ca91f349ddac17ba5e34081df4` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:15.879832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631288ca91f349ddac17ba5e34081df4)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:15.881797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:15.888406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:15.892911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5548432b584426390aa0d60abacc8cb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.893663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5548432b584426390aa0d60abacc8cb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.894341 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5548432b584426390aa0d60abacc8cb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.904546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-e1d8713a-cf26-4617-947e-31fe394acecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5548432b584426390aa0d60abacc8cb'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:15.952281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1330/2667] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:14 2022] DELETE /identity/v3/users/b5548432b584426390aa0d60abacc8cb => generated 0 bytes in 1128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:15.953838 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:15.954171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:15.954452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:15.975933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:15.977141 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:16.043992 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1331/2668] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:16.054980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:16.056384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:16.066473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:16.071117 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631288ca91f349ddac17ba5e34081df4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:16.116646 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11253c09-7f9b-4699-908e-407db1f85b59 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:16.120422 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:16.121027 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:16.121308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:16.121581 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:16.130716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:16.132182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:16.142181 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1332/2669] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:16.181772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631288ca91f349ddac17ba5e34081df4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:16.182550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631288ca91f349ddac17ba5e34081df4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:16.183232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631288ca91f349ddac17ba5e34081df4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:16.199335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631288ca91f349ddac17ba5e34081df4'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:16.218368 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1338/2670] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:15 2022] DELETE /identity/v3/users/631288ca91f349ddac17ba5e34081df4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:16.829465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:16.830315 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:16.832516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:16.833251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:16.853449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeee265-f785-446e-9a4e-595cbbadacaf service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:16.934975 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:16.941264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:16.941264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:16.941264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] PATH_INFO: `/v3/projects/b33157dbeaf84869adf98a90eb87d8a4` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:16.941264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33157dbeaf84869adf98a90eb87d8a4)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:16.941264 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:16.948519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee009b5-4709-4cd6-9555-27b94c2011d5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:16.955336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:16.956659 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:16.957146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:16.959541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] PATH_INFO: `/v3/projects/1772e196a92e49a78b88a183f5187773` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:16.961710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1772e196a92e49a78b88a183f5187773)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:16.964819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:17.098257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33157dbeaf84869adf98a90eb87d8a4', 'request_id': 'req-caae1774-9ccf-41e5-b5f8-a3537273a37d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.114690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33157dbeaf84869adf98a90eb87d8a4', 'request_id': 'req-caae1774-9ccf-41e5-b5f8-a3537273a37d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.125103 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1772e196a92e49a78b88a183f5187773', 'request_id': 'req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.135583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1772e196a92e49a78b88a183f5187773', 'request_id': 'req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.225040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33157dbeaf84869adf98a90eb87d8a4', 'request_id': 'req-caae1774-9ccf-41e5-b5f8-a3537273a37d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.238142 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] The token cache is being invalidate because project b33157dbeaf84869adf98a90eb87d8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:17.239310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.239583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-caae1774-9ccf-41e5-b5f8-a3537273a37d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.241183 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1333/2671] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:16 2022] DELETE /identity/v3/projects/b33157dbeaf84869adf98a90eb87d8a4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:17.246368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192a60-ea30-4e1c-98f6-93061b2afbba None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.246658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192a60-ea30-4e1c-98f6-93061b2afbba None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.247073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192a60-ea30-4e1c-98f6-93061b2afbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.274553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f8192a60-ea30-4e1c-98f6-93061b2afbba None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:17.276158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f8192a60-ea30-4e1c-98f6-93061b2afbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:17.313516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:17.327975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1772e196a92e49a78b88a183f5187773', 'request_id': 'req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.343545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] The token cache is being invalidate because project 1772e196a92e49a78b88a183f5187773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:17.343873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.344688 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.346518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1339/2672] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:16 2022] DELETE /identity/v3/projects/1772e196a92e49a78b88a183f5187773 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:17.351431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:17.353462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:17.355698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1334/2673] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:17.369200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.369549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.369748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.378770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b19f3f-378c-4592-b1dd-3c79cb23a2a5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:17.384573 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:17.385865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.385865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.385865 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] PATH_INFO: `/v3/users/2eb91e50f22d407d9ad61ab1c6a3ebb7` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.387181 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb91e50f22d407d9ad61ab1c6a3ebb7)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:17.389005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:17.397260 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:17.400372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:17.401878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:17.411384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1335/2674] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:17.417111 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:17.427160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340f7b-65ce-443f-94ce-9f2cb835b6aa None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:17.432182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:17.432795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.433080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.433354 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.434016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:17.434326 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:17.436101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:17.450457 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1336/2675] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:35:17.454952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:17.466263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd08f-5b7b-44e8-beeb-7b43ee459afe admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:17.472207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:17.472845 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.473125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.473399 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.474051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:17.475989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:17.633781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1337/2676] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:17 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:17.638091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:17.681817 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb91e50f22d407d9ad61ab1c6a3ebb7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.722898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf52f4-5cd8-4a69-b7b6-5ceb4298e426 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:17.731584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:17.732335 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.732621 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.732898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.733541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:17.735474 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:17.773232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb91e50f22d407d9ad61ab1c6a3ebb7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.773990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb91e50f22d407d9ad61ab1c6a3ebb7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.774742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb91e50f22d407d9ad61ab1c6a3ebb7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.789516 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb91e50f22d407d9ad61ab1c6a3ebb7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:17.804293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1340/2677] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:17 2022] DELETE /identity/v3/users/2eb91e50f22d407d9ad61ab1c6a3ebb7 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:17.887118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1338/2678] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:17 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:17.893447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:17.896000 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:17.987304 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8efbc-aeba-4897-9e37-5079dff20ed7 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:17.993633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:17.994257 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:17.994541 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:17.994824 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:17.995544 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:17.997225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.008830 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1341/2679] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:18.013631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:18.025358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0dab9-e75d-4be2-b549-6d75e3c11fd0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:18.030185 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:18.030797 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.031082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.031359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.032077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:18.033690 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1182b7-e99c-4666-a59c-8e4d47445bb5 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.045328 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1339/2680] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:18.050055 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:18.061475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471d56-544b-46ec-9d5d-c114eb403712 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:18.066872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:18.067548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.068113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.068402 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] PATH_INFO: `/v3/projects/98ca8ec56cf04b3d93c2850a1c11c076/users/d892a86a0ed2490687e9f379ed50e7bd/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.086991 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ca8ec56cf04b3d93c2850a1c11c076, user_id=d892a86a0ed2490687e9f379ed50e7bd, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:18.090540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.231548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1342/2681] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:18 2022] PUT /identity/v3/projects/98ca8ec56cf04b3d93c2850a1c11c076/users/d892a86a0ed2490687e9f379ed50e7bd/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:18.246171 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.246459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.246741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.281293 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] MFA Rules not processed for user `d892a86a0ed2490687e9f379ed50e7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:18.282549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:18.365572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1340/2682] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:18.378839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:18.381713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:18.414706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:18.414706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:18.463024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b17f7-d420-4fd0-9953-2327fed02a73 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:18.469490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:18.469490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.469490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.469490 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.477104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:18.478557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.489980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1343/2683] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:18.511930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcb686-6daa-4256-9693-58bdcf3e1459 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:18.515435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:18.517982 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:18.518700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.519508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.519912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] PATH_INFO: `/v3/projects/d7df25627381412f962a6c2e3ab4875b` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.521163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7df25627381412f962a6c2e3ab4875b)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:18.523502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.527680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8bba4-4698-4ab6-a93f-4453d3b83f40 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:18.532713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:18.533359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.533639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.533916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] PATH_INFO: `/v3/users/d892a86a0ed2490687e9f379ed50e7bd` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.535588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d892a86a0ed2490687e9f379ed50e7bd)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:18.537672 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:18.713579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7df25627381412f962a6c2e3ab4875b', 'request_id': 'req-11805e37-d848-4f66-a170-09484381fcd1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:18.720773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7df25627381412f962a6c2e3ab4875b', 'request_id': 'req-11805e37-d848-4f66-a170-09484381fcd1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:18.893057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7df25627381412f962a6c2e3ab4875b', 'request_id': 'req-11805e37-d848-4f66-a170-09484381fcd1'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:18.906330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] The token cache is being invalidate because project d7df25627381412f962a6c2e3ab4875b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:18.906804 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:18.907441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-11805e37-d848-4f66-a170-09484381fcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:18.909072 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1341/2684] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:18 2022] DELETE /identity/v3/projects/d7df25627381412f962a6c2e3ab4875b => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:18.915728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2670-b2f4-40b6-a19f-1664c12203c4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:18.916039 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2670-b2f4-40b6-a19f-1664c12203c4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:18.916426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2670-b2f4-40b6-a19f-1664c12203c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:18.943931 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-8cef2670-b2f4-40b6-a19f-1664c12203c4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:18.945187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8cef2670-b2f4-40b6-a19f-1664c12203c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:18.952342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd892a86a0ed2490687e9f379ed50e7bd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:19.034114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1342/2685] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:19.041989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.042276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.042557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.064226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:19.064723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd892a86a0ed2490687e9f379ed50e7bd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:19.067040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:19.067186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd892a86a0ed2490687e9f379ed50e7bd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:19.068218 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd892a86a0ed2490687e9f379ed50e7bd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:19.080420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1343/2686] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:19.082715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.092453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f80c8d88-f7f3-4134-861a-d67b16d458be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd892a86a0ed2490687e9f379ed50e7bd'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:19.096307 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9c22d-da7b-42d6-a25f-4ccf9bae8966 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.104161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.105132 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.105647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.105909 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.106574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:19.106895 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.111872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.112228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1344/2687] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:18 2022] DELETE /identity/v3/users/d892a86a0ed2490687e9f379ed50e7bd => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:19.114142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.114433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.114713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.124241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1344/2688] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:19.128701 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.135133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:19.136433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:19.138667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cddf78-d49f-460b-9d98-d9442bc2a225 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.199311 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1345/2689] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:19.199559 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.200218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.200500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.200777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.201407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.203642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.209510 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.210875 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:19.273460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b711a70-768b-4202-bde9-6decffdf2aea None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.277237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.277834 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.278114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.278385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.289282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.290702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29501cf7-807c-4f4a-9c33-c69ab57c7999 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.301673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1346/2690] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:19.418418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.418704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.418982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.430598 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:19.441877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:19.443115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:19.505833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1347/2691] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:19.519213 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.520716 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:19.531243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1345/2692] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:19 2022] POST /identity/v3/projects => generated 385 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:19.536058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.621976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7330c-7fc5-4431-8f80-f65cee9fd758 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.630176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768581-9cf7-462c-8035-bc4a6159b225 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.632246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.633660 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.633967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.634245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.635247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.637356 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.712350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.712985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.713272 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.713552 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.755900 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:19.773875 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1346/2693] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:19 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:19.775828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.798465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8b607-0487-4667-9490-941c9f9f1513 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.798664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.799282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.802641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.803282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.803542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.803987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.804891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.806599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.809366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1348/2694] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 19:35:19.819583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1347/2695] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:19.830347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.852686 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd5ca4-d0de-42a2-951d-d9cf50d06abb service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.861496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.861496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.861496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.861496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.863551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.865788 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:19.879721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1349/2696] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:19.884427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:19.895691 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769fc7-6bd9-4e15-b635-533aa4037eaf admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:19.900745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:19.901662 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:19.902004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:19.902380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] PATH_INFO: `/v3/projects/f4d52175d256490da4e2e66079cc951a/users/138018cc24f742c8be624438fb93b346/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:19.919998 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d52175d256490da4e2e66079cc951a, user_id=138018cc24f742c8be624438fb93b346, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:19.923079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216ae12-dc98-4f69-928a-a974115f40bf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:20.014218 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1348/2697] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:19 2022] PUT /identity/v3/projects/f4d52175d256490da4e2e66079cc951a/users/138018cc24f742c8be624438fb93b346/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:20.018300 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:20.020508 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:20.021674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.021886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.022197 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.037654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb913d56-0677-4beb-9889-6472862f1de4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:20.042098 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:20.056540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] MFA Rules not processed for user `138018cc24f742c8be624438fb93b346`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:20.058502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:20.152298 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:20.153966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.155150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.155698 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] PATH_INFO: `/v3/projects/98ca8ec56cf04b3d93c2850a1c11c076` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.157827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98ca8ec56cf04b3d93c2850a1c11c076)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:20.161106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:20.170989 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1349/2698] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:20.181872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:20.184332 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:20.252378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3535e-25b2-4177-8af1-2e7154ef7bde None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:20.256670 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:20.257291 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.257580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.257856 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.267409 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:20.269661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8b37c-3cdb-4046-aed6-cd1e586b43fe service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:20.279433 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1350/2699] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:20.318635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ca8ec56cf04b3d93c2850a1c11c076', 'request_id': 'req-42252326-421a-4f25-81d3-346a3cb5da50'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:20.336028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ca8ec56cf04b3d93c2850a1c11c076', 'request_id': 'req-42252326-421a-4f25-81d3-346a3cb5da50'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:20.427383 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ca8ec56cf04b3d93c2850a1c11c076', 'request_id': 'req-42252326-421a-4f25-81d3-346a3cb5da50'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:20.442225 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] The token cache is being invalidate because project 98ca8ec56cf04b3d93c2850a1c11c076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:20.442574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:20.443365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-42252326-421a-4f25-81d3-346a3cb5da50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:20.444902 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1350/2700] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:20 2022] DELETE /identity/v3/projects/98ca8ec56cf04b3d93c2850a1c11c076 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:20.450214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.450532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.450812 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.467318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:20.473420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:20.475441 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:20.548857 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1351/2701] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:20.559699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.560150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.560399 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.583473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:20.584681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:20.593041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1351/2702] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:20.598450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:20.609445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b1050-04ec-4a9c-b983-2073514493e4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:20.617706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:20.620766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.620766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.620766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.621089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:20.621089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:20.622840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:20.636583 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1352/2703] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:20.641396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:20.655050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325d28-c8cb-439c-9198-167f8db5b583 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:20.661053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:20.661743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.662016 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.662226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.662883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:20.664876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:20.809517 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1352/2704] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:20 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:35:20.814290 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:20.825269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e218e-d2ce-4873-908f-eab6e7010534 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:20.887905 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:20.888526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:20.888833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:20.889113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:20.889746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:20.891811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:21.035891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1353/2705] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:20 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:21.039993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:21.051501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885e8ca-850c-45fe-ab06-999b451b54ab admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:21.056196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:21.057142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:21.057436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:21.057810 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:21.061645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:21.061645 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:21.079310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1353/2706] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:21.083460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:21.096064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8dabb-3086-4555-bb7d-c56f0f4a6d1c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:21.102553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:21.102553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:21.102553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:21.102553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:21.103021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:21.105261 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e4de7-a133-4c8d-91be-9bb8dd319ac7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:21.119275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1354/2707] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:21.123561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:21.127287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:21.135072 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa32c-8bc8-408f-a781-acf346da6322 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:21.139980 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:21.140649 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:21.140935 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:21.141212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] PATH_INFO: `/v3/projects/7aa911d2360b4c568a95c05caf7c9436/users/fff1420ba7444b8da06c01f2f91c1c56/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:21.157068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa911d2360b4c568a95c05caf7c9436, user_id=fff1420ba7444b8da06c01f2f91c1c56, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:21.159754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:21.270291 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1354/2708] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:21 2022] PUT /identity/v3/projects/7aa911d2360b4c568a95c05caf7c9436/users/fff1420ba7444b8da06c01f2f91c1c56/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:21.281444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:21.281730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:21.282007 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:21.312262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] MFA Rules not processed for user `fff1420ba7444b8da06c01f2f91c1c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:21.313479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:21.373873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:21.385873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1355/2709] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:21.398014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:21.399905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:21.450980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:21.465045 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d3d9b-e431-4cbf-aace-8ff2e348ef10 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:21.468901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:21.469509 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:21.469787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:21.470060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:21.479633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:21.481195 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04d0da-063c-4551-938e-915c50fa01c5 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:21.491125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1355/2710] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:21.568193 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:23.691908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:23.695910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:23.765284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb7504-9945-4924-9bb9-957de513aaa4 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:23.765560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:23.766214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:23.766497 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:23.766781 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] PATH_INFO: `/v3/users/138018cc24f742c8be624438fb93b346` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:23.768200 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138018cc24f742c8be624438fb93b346)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:23.770664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:24.021068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138018cc24f742c8be624438fb93b346'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:24.161795 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138018cc24f742c8be624438fb93b346'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:24.162524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138018cc24f742c8be624438fb93b346'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:24.163158 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138018cc24f742c8be624438fb93b346'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:24.171562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138018cc24f742c8be624438fb93b346'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:24.189714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1356/2711] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:23 2022] DELETE /identity/v3/users/138018cc24f742c8be624438fb93b346 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:24.194253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f4849-d2de-4f77-b1b5-bfe3e414fe14 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.194713 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f4849-d2de-4f77-b1b5-bfe3e414fe14 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.194831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f4849-d2de-4f77-b1b5-bfe3e414fe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.213777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-1c6f4849-d2de-4f77-b1b5-bfe3e414fe14 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:24.214959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1c6f4849-d2de-4f77-b1b5-bfe3e414fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:24.274829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1356/2712] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:24.285246 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:24.286725 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:24.293330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:24.343995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4000e9-b7c7-4377-8902-c2b731af7604 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:24.348329 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:24.348935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.349217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.349493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.358508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:24.360050 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6087f-9b04-4eaf-a640-2113c4ad38b7 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:24.370538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1357/2713] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:24.688278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc3be3-0957-46ab-8766-69fcec02cc3b None None] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.688652 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc3be3-0957-46ab-8766-69fcec02cc3b None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.689245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc3be3-0957-46ab-8766-69fcec02cc3b None None] PATH_INFO: `/` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.691358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1357/2714] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:35:24.697104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48919109-684b-41ef-9431-13813ee4bc0c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.697916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48919109-684b-41ef-9431-13813ee4bc0c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.697916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48919109-684b-41ef-9431-13813ee4bc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.718694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-48919109-684b-41ef-9431-13813ee4bc0c None None] MFA Rules not processed for user `ed6b2414b84f4ba0a59b3700b122ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:24.719934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-48919109-684b-41ef-9431-13813ee4bc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:24.728057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1358/2715] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:35:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:24.741200 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc3be3-0957-46ab-8766-69fcec02cc3b None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:24.752759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc3be3-0957-46ab-8766-69fcec02cc3b None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:24.757028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:24.757799 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.758087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.758360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.766486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:24.768137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37b592-9a52-4764-8a21-238f3230d188 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:24.776938 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1358/2716] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:24.865223 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.865514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.865848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.867214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1359/2717] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:35:24.872963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.880100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.880100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.893853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:24.895070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:24.939973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:24.942902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:24.969086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc6bcf-f1a4-4c69-bf89-9545249f9e2f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:24.973675 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1359/2718] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:35:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:24.973971 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:24.974673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:24.974957 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:24.975232 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] PATH_INFO: `/v3/projects/f4d52175d256490da4e2e66079cc951a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:24.978720 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d52175d256490da4e2e66079cc951a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:24.980953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:24.984925 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:24.995848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699e2ee-d908-4112-9905-e91391f0790c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:24.999746 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.000359 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.000676 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.000962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.013747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.015209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1752ed-d6e5-4db3-9be3-961000f417fb service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.026586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1360/2719] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:25.087160 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d52175d256490da4e2e66079cc951a', 'request_id': 'req-53aed5c9-41aa-4887-81e3-756be688f876'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:25.094041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d52175d256490da4e2e66079cc951a', 'request_id': 'req-53aed5c9-41aa-4887-81e3-756be688f876'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:25.164719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d52175d256490da4e2e66079cc951a', 'request_id': 'req-53aed5c9-41aa-4887-81e3-756be688f876'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:25.177154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] The token cache is being invalidate because project f4d52175d256490da4e2e66079cc951a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:25.177449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:25.178193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-53aed5c9-41aa-4887-81e3-756be688f876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:25.179616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1360/2720] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:24 2022] DELETE /identity/v3/projects/f4d52175d256490da4e2e66079cc951a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:25.184534 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.184828 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.185108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.207369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:25.208677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:25.233907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:25.270470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1361/2721] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:25.279265 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.279552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.279866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.301616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:25.306708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:25.310707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1361/2722] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:25.315246 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.325723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd5150-4f1b-43f0-ac53-a852d217adfb None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.331219 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.332409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.332687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.332957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.333605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:25.333911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.335738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.347736 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1362/2723] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:25.352625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.361811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0e980-329d-40ce-be70-93b4579ca798 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.366858 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.367460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.367853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.368131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.368768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.370744 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.482675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1362/2724] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:25 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:25.486993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.497733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5a2a2-b50b-4b24-ac6d-b0fe2564a31d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.558133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.558738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.559019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.559292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.560080 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.561983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.670638 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1363/2725] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:25 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:25.675054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.686751 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce0751-0b51-4422-911d-d631a203522a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.692588 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.693144 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.693423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.693695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.694336 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.695999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.706695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1363/2726] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:25.710764 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.722519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667574cf-8745-4690-acda-adbf3c91c392 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.727561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.728202 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.728486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.728760 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.729427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.731115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2e15-5b79-4410-956b-caab8bfd520e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.746635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1364/2727] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:25.751480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:25.762606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0104e4-03f5-48f4-92cf-2446d57c25aa admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:25.767472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:25.768177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.768461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.768736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] PATH_INFO: `/v3/projects/a42a0c09878c4a73a516209a9c0af3fa/users/b3a17f229d1142da9034ec2d3b2da444/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.784397 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42a0c09878c4a73a516209a9c0af3fa, user_id=b3a17f229d1142da9034ec2d3b2da444, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:25.787101 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:25.894801 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1364/2728] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:25 2022] PUT /identity/v3/projects/a42a0c09878c4a73a516209a9c0af3fa/users/b3a17f229d1142da9034ec2d3b2da444/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:25.967071 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:25.967350 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:25.967629 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:25.996869 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] MFA Rules not processed for user `b3a17f229d1142da9034ec2d3b2da444`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:25.998086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:26.067380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1365/2729] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:26.078528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:26.085449 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:26.102615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:26.139193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594ad07-def2-4e25-85f9-cd975271f29b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:26.142936 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:26.143551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:26.143863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:26.144140 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:26.153552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:26.155025 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6641f3-b91b-4639-982a-5f22c4df51c6 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:26.165155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1365/2730] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:26.191446 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:26.219721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:26.256049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17baffb-bd16-4ded-abc9-a6f75538af80 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:26.260589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:26.261221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:26.261502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:26.261781 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] PATH_INFO: `/v3/users/b3a17f229d1142da9034ec2d3b2da444` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:26.262922 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a17f229d1142da9034ec2d3b2da444)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:26.265018 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:26.443990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a17f229d1142da9034ec2d3b2da444'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:26.532610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a17f229d1142da9034ec2d3b2da444'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:26.533256 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a17f229d1142da9034ec2d3b2da444'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:26.533883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a17f229d1142da9034ec2d3b2da444'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:26.548066 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a17f229d1142da9034ec2d3b2da444'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:26.560397 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1366/2731] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:26 2022] DELETE /identity/v3/users/b3a17f229d1142da9034ec2d3b2da444 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:26.565273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:26.565560 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:26.565835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:26.585613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:26.586889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:26.647995 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1366/2732] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:26.657826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:26.659259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:26.715839 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709ee65-1fe8-497f-9d3e-3659ff9b57bf admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:26.719450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:26.720086 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:26.720370 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:26.720644 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:26.729613 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:26.731033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:26.740704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1367/2733] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:27.249167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:27.250898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:27.263704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678d94-268c-4235-9d5c-c8fecdf8002f None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:27.268302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:27.268959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:27.269227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:27.269503 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] PATH_INFO: `/v3/projects/a42a0c09878c4a73a516209a9c0af3fa` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:27.270616 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42a0c09878c4a73a516209a9c0af3fa)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:27.272558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:27.393538 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42a0c09878c4a73a516209a9c0af3fa', 'request_id': 'req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:27.399707 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42a0c09878c4a73a516209a9c0af3fa', 'request_id': 'req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:27.495573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42a0c09878c4a73a516209a9c0af3fa', 'request_id': 'req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:27.507215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] The token cache is being invalidate because project a42a0c09878c4a73a516209a9c0af3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:27.507506 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:27.508172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-28a3dd87-f516-4c50-871f-4fc6f2cd8fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:27.509695 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1367/2734] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:27 2022] DELETE /identity/v3/projects/a42a0c09878c4a73a516209a9c0af3fa => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:32.052928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:32.054338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:32.067030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed9bea-5fdc-4433-9b91-51d13af3ca84 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:32.111030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:32.124468 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:32.125104 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:32.125640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:32.126008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] PATH_INFO: `/v3/users/fff1420ba7444b8da06c01f2f91c1c56` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:32.126856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff1420ba7444b8da06c01f2f91c1c56)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:32.128807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:32.309323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff1420ba7444b8da06c01f2f91c1c56'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:32.427160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff1420ba7444b8da06c01f2f91c1c56'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:32.427836 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff1420ba7444b8da06c01f2f91c1c56'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:32.428456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff1420ba7444b8da06c01f2f91c1c56'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:32.442259 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff1420ba7444b8da06c01f2f91c1c56'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:32.454467 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1368/2735] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:32 2022] DELETE /identity/v3/users/fff1420ba7444b8da06c01f2f91c1c56 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:32.459432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:32.459943 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:32.460049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:32.480303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:32.481514 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:32.522182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:32.540076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1368/2736] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:32.550499 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:32.551959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:32.618111 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc619e9a-ec5c-4c35-9bef-91b78a36425e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:32.621744 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:32.622342 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:32.622669 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:32.622990 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:32.632565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:32.633986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c53c-bab1-47e1-ac17-5583bbfa8712 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:32.644812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1369/2737] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:33.202003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:33.202167 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:33.214698 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb2895-e59c-4253-8b5b-4bd5ace5bcc7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:33.219510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:33.220242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.220528 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.220802 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] PATH_INFO: `/v3/projects/7aa911d2360b4c568a95c05caf7c9436` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.222477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa911d2360b4c568a95c05caf7c9436)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:33.224703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:33.382784 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa911d2360b4c568a95c05caf7c9436', 'request_id': 'req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:33.389346 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa911d2360b4c568a95c05caf7c9436', 'request_id': 'req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:33.478508 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa911d2360b4c568a95c05caf7c9436', 'request_id': 'req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:33.490584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] The token cache is being invalidate because project 7aa911d2360b4c568a95c05caf7c9436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:33.490885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:33.491769 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-57073d60-b30e-4b9e-83e2-91d5ddbe7d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:33.493317 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1369/2738] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:33 2022] DELETE /identity/v3/projects/7aa911d2360b4c568a95c05caf7c9436 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:33.498235 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.498735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.498851 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.518531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:33.519846 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:33.580807 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1370/2739] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:33.588663 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.588948 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.589221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.609817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:33.611092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:33.619713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1370/2740] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:33.624024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:33.633735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7145c-2671-4e18-a926-43d315bbf462 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:33.638336 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:33.638929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.639208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.639485 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.640248 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:33.640553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:33.642168 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:33.646897 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:33.656384 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1371/2741] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:33.670913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:33.680796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5cad3-303c-4d51-a31c-e5eb3146a8d7 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:33.685846 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:33.686444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.686721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.686991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.687627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:33.689867 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:33.786819 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1371/2742] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:33 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:33.790993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:33.801486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd3235-916f-4edd-a928-64a80fa9e037 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:33.861382 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:33.861997 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.862277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.862549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.863198 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:33.865112 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:33.967953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1372/2743] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:33 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:33.971938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:33.982828 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbb1fb-25f1-4a88-a1ce-4db16b53bd77 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:33.987493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:33.988117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:33.988401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:33.988675 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:33.989313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:33.990930 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:34.000763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1372/2744] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:35:34.004683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:34.014970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c165c-85a0-49a9-8dc8-fca701b9beeb admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:34.020161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:34.020759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:34.021044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:34.021321 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:34.021962 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:34.023548 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c010d9-7e9d-4811-9fb6-c095b55f516d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:34.034138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1373/2745] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:35:34.038919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:34.048919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b1919-64db-49bf-9850-beed30442a16 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:34.053569 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:34.054221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:34.054502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:34.054833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] PATH_INFO: `/v3/projects/a63384284dee4152bdb6a4d93a144416/users/c6201918e6ef448eb479853340480855/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:34.070553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63384284dee4152bdb6a4d93a144416, user_id=c6201918e6ef448eb479853340480855, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:34.073296 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:34.150713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1373/2746] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:35:34 2022] PUT /identity/v3/projects/a63384284dee4152bdb6a4d93a144416/users/c6201918e6ef448eb479853340480855/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:34.218390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:34.218954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:34.219338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:34.248677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] MFA Rules not processed for user `c6201918e6ef448eb479853340480855`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:34.249986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:34.318395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1374/2747] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:34.329480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:34.331162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:34.389013 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0ce34-0a85-4c8b-8d85-2c41586fa1b0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:34.392718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:34.393314 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:34.393596 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:34.393866 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:34.402740 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:34.404193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c5ea7-c5b0-48a7-9c45-bd757f39339a service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:34.413096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1374/2748] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:35:44.314859 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:44.409374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:57.972795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:58.045069 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aecab-3bc4-4fdf-9a7b-e8827b61dc30 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:58.049773 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:58.050411 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:58.050696 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:58.050972 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] PATH_INFO: `/v3/users/c6201918e6ef448eb479853340480855` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:58.052212 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6201918e6ef448eb479853340480855)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:58.054141 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:58.303831 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6201918e6ef448eb479853340480855'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:58.456177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6201918e6ef448eb479853340480855'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:58.457082 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6201918e6ef448eb479853340480855'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:58.458154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6201918e6ef448eb479853340480855'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:58.480240 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6201918e6ef448eb479853340480855'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:58.493873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1375/2749] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:35:57 2022] DELETE /identity/v3/users/c6201918e6ef448eb479853340480855 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:35:58.499331 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:58.500178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:58.500178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:58.525940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:58.527918 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:58.628193 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1375/2750] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:35:58.638279 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:58.639883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:58.703523 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4b0e6-3893-47fc-b5c7-f823df216df4 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:58.707236 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:58.707818 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:58.708149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:58.708945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:58.719470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:58.721232 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6da96-8e0e-42dc-a76b-8671c454d035 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:58.732589 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1376/2751] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:35:59.263233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:59.264885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:59.276787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b6c6-785f-4cf5-a3c2-a0eb5b743b1a None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:59.281782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:59.282419 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.282704 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.282981 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] PATH_INFO: `/v3/projects/a63384284dee4152bdb6a4d93a144416` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.284194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63384284dee4152bdb6a4d93a144416)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:59.286096 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:59.440415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63384284dee4152bdb6a4d93a144416', 'request_id': 'req-a642909a-30d5-4a28-878a-f5126ffa365e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:59.447339 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63384284dee4152bdb6a4d93a144416', 'request_id': 'req-a642909a-30d5-4a28-878a-f5126ffa365e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:59.523906 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63384284dee4152bdb6a4d93a144416', 'request_id': 'req-a642909a-30d5-4a28-878a-f5126ffa365e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:59.535468 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] The token cache is being invalidate because project a63384284dee4152bdb6a4d93a144416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:35:59.535794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:59.536962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-a642909a-30d5-4a28-878a-f5126ffa365e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:35:59.538459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1376/2752] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:35:59 2022] DELETE /identity/v3/projects/a63384284dee4152bdb6a4d93a144416 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:35:59.543420 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.543931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.544044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.564782 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:59.566010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:59.583978 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:35:59.630556 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1377/2753] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:59.638723 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.639017 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.639299 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.660041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:35:59.661263 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:35:59.669869 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1377/2754] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:35:59.674190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:59.685522 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc059f-a2cf-4431-82a8-12818e3682f6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:59.690462 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:59.691070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.691353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.691635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.692333 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:35:59.692621 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:59.695586 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:59.711952 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1378/2755] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:35:59.717079 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:59.726829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7a815-7e1f-4250-99a1-763968e074af None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:59.731584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:59.732206 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.732485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.732782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.733406 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:59.735277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:35:59.825997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1378/2756] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:35:59 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:35:59.830353 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:35:59.841283 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb53c0-a03b-4e0f-9cd6-348f01209e43 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:35:59.905174 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:35:59.905809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:35:59.906096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:35:59.906373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:35:59.907009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:35:59.908970 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:00.022089 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1379/2757] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:35:59 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:36:00.026355 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:00.037535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376075d-ccae-4521-824e-81cbac1223ef admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:00.042083 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:00.042703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:00.042985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:00.043259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:00.043927 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:00.045525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:00.055615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1379/2758] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:36:00.059841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:00.070354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617c3fc-b3d0-4a4c-8633-7ade83125791 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:00.075007 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:00.075592 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:00.075911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:00.076241 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:00.076827 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:00.078426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db64bc-286a-4d51-92c3-1db10afd6790 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:00.090445 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1380/2759] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:36:00.094783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:00.108321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9e596-d482-4b2d-abed-dd0cc5b73950 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:00.114063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:00.114721 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:00.115003 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:00.115278 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] PATH_INFO: `/v3/projects/d5ac85014f084b96b9ba733cf0361de7/users/de98c1dfeee144f4b92a6725f581b00b/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:00.130631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ac85014f084b96b9ba733cf0361de7, user_id=de98c1dfeee144f4b92a6725f581b00b, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:00.133473 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:00.235946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1380/2760] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:36:00 2022] PUT /identity/v3/projects/d5ac85014f084b96b9ba733cf0361de7/users/de98c1dfeee144f4b92a6725f581b00b/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:36:00.247881 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d0958-fdbd-42a2-b7f5-014a668818f6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:00.248160 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d0958-fdbd-42a2-b7f5-014a668818f6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:00.248438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d0958-fdbd-42a2-b7f5-014a668818f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:00.291770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-518d0958-fdbd-42a2-b7f5-014a668818f6 None None] MFA Rules not processed for user `de98c1dfeee144f4b92a6725f581b00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:36:00.291770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-518d0958-fdbd-42a2-b7f5-014a668818f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:00.358702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1381/2761] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:36:00.369980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:00.372019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:00.395258 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:36:00.430644 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efed9b-dfa7-42e5-9f07-d4844f147172 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:00.434284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:00.434891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:00.435175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:00.435452 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:00.444860 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:00.446394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:00.456615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1381/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:36:08.757568 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892141d-5a94-44b1-8313-f95225f8c1a6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:08.758068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892141d-5a94-44b1-8313-f95225f8c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:08.758068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892141d-5a94-44b1-8313-f95225f8c1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:08.778650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-0892141d-5a94-44b1-8313-f95225f8c1a6 None None] MFA Rules not processed for user `250673e17f40455ca2c5e0036f5a9e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:36:08.780063 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-0892141d-5a94-44b1-8313-f95225f8c1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:08.792849 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:36:08.838879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1382/2763] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:36:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:36:10.515367 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=17 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:36:19.104296 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=17 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:36:42.023896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:42.087493 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9f59-377e-4f97-a3c4-cc22fc971a4d service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:42.092234 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:42.092864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:42.093152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:42.093432 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] PATH_INFO: `/v3/users/de98c1dfeee144f4b92a6725f581b00b` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:42.094636 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de98c1dfeee144f4b92a6725f581b00b)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:42.096602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:42.348612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de98c1dfeee144f4b92a6725f581b00b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:42.460766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de98c1dfeee144f4b92a6725f581b00b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:42.461450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de98c1dfeee144f4b92a6725f581b00b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:42.462057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de98c1dfeee144f4b92a6725f581b00b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:42.470252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de98c1dfeee144f4b92a6725f581b00b'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:42.488152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1382/2764] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:36:42 2022] DELETE /identity/v3/users/de98c1dfeee144f4b92a6725f581b00b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:36:42.492647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721641fa-b973-403e-be33-de06b429bac0 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:42.493176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721641fa-b973-403e-be33-de06b429bac0 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:42.493176 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721641fa-b973-403e-be33-de06b429bac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:42.513631 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-721641fa-b973-403e-be33-de06b429bac0 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:36:42.514855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-721641fa-b973-403e-be33-de06b429bac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:42.576159 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1383/2765] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:36:42.585973 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:42.587404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:42.649081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e754e-d0c8-4287-877f-64f73d0c5817 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:42.652661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:42.653275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:42.654117 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:42.654391 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:42.663608 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:42.665063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:42.676176 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1383/2766] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:36:43.113044 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721641fa-b973-403e-be33-de06b429bac0 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:36:43.114531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-721641fa-b973-403e-be33-de06b429bac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:36:43.129465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721641fa-b973-403e-be33-de06b429bac0 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:36:43.134988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:36:43.135683 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:36:43.136122 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:36:43.136237 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] PATH_INFO: `/v3/projects/d5ac85014f084b96b9ba733cf0361de7` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:36:43.137514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ac85014f084b96b9ba733cf0361de7)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:36:43.139955 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:36:43.256009 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ac85014f084b96b9ba733cf0361de7', 'request_id': 'req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:43.262876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ac85014f084b96b9ba733cf0361de7', 'request_id': 'req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:43.340612 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ac85014f084b96b9ba733cf0361de7', 'request_id': 'req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:43.352119 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] The token cache is being invalidate because project d5ac85014f084b96b9ba733cf0361de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:36:43.352409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:43.353056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:36:43.354715 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1384/2767] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:36:43 2022] DELETE /identity/v3/projects/d5ac85014f084b96b9ba733cf0361de7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:36:52.670902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:36:53.348673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:04.808794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:04.823539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd42df-4b39-4f0c-a649-248a8052b6eb service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:04.878423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:04.879267 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:04.879642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:04.880049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:04.882190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:04.958926 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:04.960415 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:04.972987 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1384/2768] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:08.912988 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:08.914513 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:08.932149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6f4cf-35bd-45aa-a2da-9d4ddd6b8a85 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:08.937737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:08.938351 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:08.938635 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:08.938905 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:08.941172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:09.004026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:09.005643 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:09.014928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1385/2769] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:09.500207 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:09.501728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:09.512630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab54903-7308-456a-9866-32f365ab9905 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:09.526898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:09.526898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:09.526898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:09.526898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:09.527359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:09.584972 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:09.586411 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:09.596413 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1385/2770] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:37:13.416708 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:13.418234 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:13.506787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad0c9-ae51-4f99-9d4b-3a85dac37be4 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:13.511880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:13.512483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:13.512765 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:13.513043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:13.515610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:13.527616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:13.529275 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:13.539577 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1386/2771] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:17.150086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:17.151630 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:17.171580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6830abd-9e1d-4a1e-a36b-65bd601ddec1 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:17.175952 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:17.176601 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:17.176879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:17.177152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] PATH_INFO: `/v3/users/b638c6d935e64be6bce4ecc1fb8f26ec` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:17.178384 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b638c6d935e64be6bce4ecc1fb8f26ec)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:17.180557 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:17.386565 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:17.400045 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b638c6d935e64be6bce4ecc1fb8f26ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.477126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b638c6d935e64be6bce4ecc1fb8f26ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.477890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b638c6d935e64be6bce4ecc1fb8f26ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.478624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b638c6d935e64be6bce4ecc1fb8f26ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.487172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b638c6d935e64be6bce4ecc1fb8f26ec'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.505170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1386/2772] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:37:17 2022] DELETE /identity/v3/users/b638c6d935e64be6bce4ecc1fb8f26ec => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:37:17.751243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:17.762220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426f824-1165-4c88-a562-f6e181db93c2 service placement] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:17.819681 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:17.820395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:17.820699 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:17.820959 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] PATH_INFO: `/v3/projects/a7bbec449c0142f0b910956bacedbe5e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:17.822133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bbec449c0142f0b910956bacedbe5e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:17.824161 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:17.926721 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:17.940322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bbec449c0142f0b910956bacedbe5e', 'request_id': 'req-9dae6d40-a166-4eee-9454-440d00769e3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:17.946874 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bbec449c0142f0b910956bacedbe5e', 'request_id': 'req-9dae6d40-a166-4eee-9454-440d00769e3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.012729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bbec449c0142f0b910956bacedbe5e', 'request_id': 'req-9dae6d40-a166-4eee-9454-440d00769e3d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.024429 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] The token cache is being invalidate because project a7bbec449c0142f0b910956bacedbe5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:37:18.024727 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.025403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.031576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1387/2773] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:37:17 2022] DELETE /identity/v3/projects/a7bbec449c0142f0b910956bacedbe5e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:37:18.035967 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:18.037469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:18.051448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e13965-a71a-426c-b95d-b04425fc7225 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:18.108478 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:18.109428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:18.109706 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:18.109978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] PATH_INFO: `/v3/users/a598e93e631b4cbcaac7a047fb1d5caa` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:18.111109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a598e93e631b4cbcaac7a047fb1d5caa)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:18.113092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:18.294711 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a598e93e631b4cbcaac7a047fb1d5caa'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.385479 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a598e93e631b4cbcaac7a047fb1d5caa'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.386109 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a598e93e631b4cbcaac7a047fb1d5caa'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.386719 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a598e93e631b4cbcaac7a047fb1d5caa'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.394552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-64672815-1243-490d-bbdb-55cf5b737ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a598e93e631b4cbcaac7a047fb1d5caa'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.412885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1387/2774] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:37:18 2022] DELETE /identity/v3/users/a598e93e631b4cbcaac7a047fb1d5caa => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:37:18.674848 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:18.676380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:18.692209 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae6d40-a166-4eee-9454-440d00769e3d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:18.750220 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:18.750862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:18.751152 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:18.751430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] PATH_INFO: `/v3/projects/19f7aac894e942ce8e3eb07d97871f1b` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:18.752709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f7aac894e942ce8e3eb07d97871f1b)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:18.754718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:18.877375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f7aac894e942ce8e3eb07d97871f1b', 'request_id': 'req-901334b1-a947-4878-9f48-fd83a030a115'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:18.883528 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f7aac894e942ce8e3eb07d97871f1b', 'request_id': 'req-901334b1-a947-4878-9f48-fd83a030a115'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:19.006737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f7aac894e942ce8e3eb07d97871f1b', 'request_id': 'req-901334b1-a947-4878-9f48-fd83a030a115'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:19.019136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] The token cache is being invalidate because project 19f7aac894e942ce8e3eb07d97871f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:37:19.019431 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:19.020106 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:37:19.021604 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1388/2775] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:37:18 2022] DELETE /identity/v3/projects/19f7aac894e942ce8e3eb07d97871f1b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:37:19.423225 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fe276-7d61-4335-b408-1f26f5637fb0 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:19.423820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fe276-7d61-4335-b408-1f26f5637fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:19.423950 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fe276-7d61-4335-b408-1f26f5637fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:19.470104 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:19.479896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-3f5fe276-7d61-4335-b408-1f26f5637fb0 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:37:19.481155 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f5fe276-7d61-4335-b408-1f26f5637fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:19.559881 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1388/2776] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:37:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:37:19.708281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:19.709703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:19.768918 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:19.813808 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901334b1-a947-4878-9f48-fd83a030a115 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:19.819325 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:19.819963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:19.820250 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:19.820525 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:19.829019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:19.831154 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0696-562e-4130-8ec3-91db442eaf58 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:19.839572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1389/2777] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:37:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:37:29.606794 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=28 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:29.870143 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=19 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:36.529280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:36.529868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:36.529868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] PATH_INFO: `/` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:36.531182 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1389/2778] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 15 19:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 19:37:36.536390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:36.536943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:36.536943 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:36.581027 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] MFA Rules not processed for user `c9b27eb952a04c46bb36653fb79abe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:37:36.582287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:36.655919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1390/2779] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 15 19:37:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:37:36.667564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:36.669302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:36.730284 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038162e-4b56-465e-bbec-9dcf80b4367b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:36.734603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37e3dbce9a544a519e965dc35cd5ba9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:36.735386 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:36.735713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:36.736097 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:36.747272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:36.749404 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:36.761549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1390/2780] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:37.104648 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:37.106594 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:37.190584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d7017-c2f6-4a19-a975-d3e92697e3b1 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:37.196352 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:37.196667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:37.196928 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:37.197206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:37.198989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:37.212032 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:37.213735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac7d58-a529-4432-bf9a-5166e3b5323c service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:37.226242 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1391/2781] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:38.445435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:38.446773 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:38.503089 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b7cc3-5f08-43cd-9405-e3d0760fa534 service glance] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:38.506647 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:38.507320 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:38.507780 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:38.508061 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:38.509898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:38.520477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:38.521896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:38.529913 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1391/2782] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:39.414495 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:39.415047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:39.415047 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:39.434627 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:37:39.435960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:39.443503 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1392/2783] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:37:39.456310 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:39.468768 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140a4d5-88d5-4125-9d72-0e3c371d00cb service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:39.473945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:39.474536 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:39.474822 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:39.475093 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:39.483163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:39.484741 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692c31-06f9-4e36-837b-f6fe413865ec service swift] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:39.492516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1392/2784] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:37:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:37:49.429873 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=72 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:49.489394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=58 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:37:50.530302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:50.531780 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:50.547977 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88370d-ca79-4f90-81a8-d81ed6451000 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:50.558919 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:50.559536 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:50.559853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:50.560133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:50.561912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:50.575438 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:50.577571 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:50.588570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1393/2785] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:37:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:37:52.485689 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:52.486231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:52.486231 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:52.505674 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] MFA Rules not processed for user `a2a29c5a3e2a43bb94c0bd21b33c9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:37:52.506888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:37:52.514086 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1393/2786] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 15 19:37:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:37:52.526892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:37:52.539028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13186d02-e3b5-4d8c-94db-59dc7eb07db7 service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:37:52.544189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0c4e352d9914c3a9a5225439274d91d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:37:52.544777 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:37:52.545059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:37:52.545330 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:37:52.553838 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:37:52.556540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:37:52.564287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1394/2787] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 15 19:37:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 19:38:02.500119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:02.560308 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=12 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:25.414287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:25.415890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:25.472746 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e35fe4-3b16-4034-abb4-ed0909a58aa2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:25.476303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:25.476910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:25.477194 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:25.477471 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:25.479211 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:25.536693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:25.538120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:25.546979 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1394/2788] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:38:28.220210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:28.232735 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4e302-1c1b-4604-b925-cd8287a664e6 service swift] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:28.238579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b27eb952a04c46bb36653fb79abe3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:28.239286 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:28.239820 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:28.240060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:28.241934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:28.255790 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:28.257385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:28.269389 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1395/2789] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:38:33.424614 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:33.434669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b632-6050-4ad6-aec2-c218af2fde98 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:33.438319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:33.438920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:33.439204 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:33.439480 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:33.441248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:33.504993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:33.506628 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:33.516407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1395/2790] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:38:38.191408 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:38.193170 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:38.207114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80644-ac2f-431b-8cc5-57f50556f38d service nova] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:38.213043 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:38.217133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:38.217133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:38.217133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] PATH_INFO: `/v3/users/250673e17f40455ca2c5e0036f5a9e3c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:38.217133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] RBAC: Authorizing `identity:delete_user(user_id=250673e17f40455ca2c5e0036f5a9e3c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:38.217951 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:38.559787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250673e17f40455ca2c5e0036f5a9e3c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:38.657668 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250673e17f40455ca2c5e0036f5a9e3c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:38.658293 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250673e17f40455ca2c5e0036f5a9e3c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:38.658932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250673e17f40455ca2c5e0036f5a9e3c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:38.672282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-80f7e69a-7456-411c-bfad-8f920fccfece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250673e17f40455ca2c5e0036f5a9e3c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:38.684711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1396/2791] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:38:38 2022] DELETE /identity/v3/users/250673e17f40455ca2c5e0036f5a9e3c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:38.980564 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:38.991785 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfbdb-8ab3-4548-882a-fc10f920d65a service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:39.041777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:39.053562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:39.054179 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.054459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.054732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] PATH_INFO: `/v3/projects/427b697da1034408811e994e22d4f592` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.055920 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=427b697da1034408811e994e22d4f592)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:39.057988 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:39.254836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427b697da1034408811e994e22d4f592', 'request_id': 'req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:39.261510 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427b697da1034408811e994e22d4f592', 'request_id': 'req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:39.373627 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427b697da1034408811e994e22d4f592', 'request_id': 'req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:39.384829 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] The token cache is being invalidate because project 427b697da1034408811e994e22d4f592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:38:39.385116 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:39.385749 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-fd7e9c61-2a1d-436b-8d32-590dceb9a9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:39.387758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1396/2792] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:38:38 2022] DELETE /identity/v3/projects/427b697da1034408811e994e22d4f592 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:39.393442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.394302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.394487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.415832 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:39.417051 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:39.489185 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1397/2793] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:39.498580 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.498863 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.499138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.522774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:39.522774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:39.530731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1397/2794] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:39.535245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:39.545031 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8353-02cd-471e-b438-26e5e41b86ba None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:39.550303 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:39.550941 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.551226 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.551504 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.552196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:38:39.552505 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:39.554148 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:39.567251 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1398/2795] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:38:39.571976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:39.582043 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb57774-cb1b-481b-9b49-05264df953d6 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:39.586767 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:39.587673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.587673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.587962 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.588635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:39.590533 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:39.781374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1398/2796] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:38:39 2022] POST /identity/v3/projects => generated 363 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:38:39.785492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:39.795812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bbf7e-e859-4c55-af79-3158ab5dc12f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:39.841301 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:39.854020 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:39.869138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:39.869138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:39.869138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:39.869138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:39.869138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:40.050912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1399/2797] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:38:39 2022] POST /identity/v3/users => generated 282 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:38:40.055170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:40.066996 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878994-3c3f-473b-aaaa-6ff2eac1c871 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:40.072540 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:40.073154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:40.073439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:40.073782 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:40.074590 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:40.076340 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:40.088248 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1399/2798] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:38:40.092579 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:40.103475 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b746-2d2f-4081-9d36-9fe87e54a804 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:40.108430 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:40.109570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:40.109857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:40.110138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:40.110783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:40.112471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a10d2b-0fac-4c66-90af-049622f1649d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:40.123210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1400/2799] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:38:40.128253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:40.138584 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d6784-81a4-47bd-9228-dc7031aa5ee8 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:40.152279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:40.152279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:40.152279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:40.152279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] PATH_INFO: `/v3/projects/c978f729832e4d7bbe4612a36cd32c80/users/ad07f6b2767348c095bfa536b229f69f/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:40.159923 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c978f729832e4d7bbe4612a36cd32c80, user_id=ad07f6b2767348c095bfa536b229f69f, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:40.162562 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:40.350717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1400/2800] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:38:40 2022] PUT /identity/v3/projects/c978f729832e4d7bbe4612a36cd32c80/users/ad07f6b2767348c095bfa536b229f69f/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:40.361917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:40.362196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:40.362581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:40.390491 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] MFA Rules not processed for user `ad07f6b2767348c095bfa536b229f69f`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:40.400538 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:40.459945 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1401/2801] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:40.471410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:40.473254 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:40.535764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84495a74-4b30-488e-aee7-94222ff7a192 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:40.539464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:40.540108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:40.540396 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:40.540676 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:40.550407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:40.551910 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28be0b-2ec9-45c2-9941-f83dbcb4d4f4 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:40.560793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1401/2802] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:38:50.457121 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:50.556717 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:50.818877 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:50.882022 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccfb58-d407-4f09-972c-7a23be27e04c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:50.886783 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:50.887422 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:50.887748 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:50.888029 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] PATH_INFO: `/v3/users/ad07f6b2767348c095bfa536b229f69f` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:50.889188 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad07f6b2767348c095bfa536b229f69f)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:50.891100 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:51.079449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad07f6b2767348c095bfa536b229f69f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:51.207484 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad07f6b2767348c095bfa536b229f69f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:51.208292 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad07f6b2767348c095bfa536b229f69f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:51.209116 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad07f6b2767348c095bfa536b229f69f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:51.223245 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad07f6b2767348c095bfa536b229f69f'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:51.235819 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1402/2803] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:38:50 2022] DELETE /identity/v3/users/ad07f6b2767348c095bfa536b229f69f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:51.240374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:51.240661 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:51.240937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:51.260120 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:51.261321 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:51.322799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1402/2804] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:51.335700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:51.337060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:51.400146 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5773d-007f-4f96-87f3-ade9995eec9b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:51.404048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:51.404651 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:51.404948 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:51.405204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:51.414733 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:51.416205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f532-0625-4df2-b02c-60ee850c143f service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:51.427337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1403/2805] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:38:51.859825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:51.861269 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:51.874058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a8b96-e14e-4712-92e7-8a7f2819cfc2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:51.879823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:51.880462 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:51.880758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:51.881014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] PATH_INFO: `/v3/projects/c978f729832e4d7bbe4612a36cd32c80` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:51.882671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c978f729832e4d7bbe4612a36cd32c80)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:51.884908 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.021834 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c978f729832e4d7bbe4612a36cd32c80', 'request_id': 'req-8a5ec47b-a32d-421e-bbae-f54650e45d63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:52.028063 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c978f729832e4d7bbe4612a36cd32c80', 'request_id': 'req-8a5ec47b-a32d-421e-bbae-f54650e45d63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:52.138459 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c978f729832e4d7bbe4612a36cd32c80', 'request_id': 'req-8a5ec47b-a32d-421e-bbae-f54650e45d63'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:52.149679 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] The token cache is being invalidate because project c978f729832e4d7bbe4612a36cd32c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:38:52.156732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:52.156732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8a5ec47b-a32d-421e-bbae-f54650e45d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:38:52.156732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1403/2806] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:38:51 2022] DELETE /identity/v3/projects/c978f729832e4d7bbe4612a36cd32c80 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:52.157050 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.157498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.157615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.178732 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:52.179965 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:52.197664 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:52.242280 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1404/2807] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:38:52.250573 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.250871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.251154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.272420 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:52.273659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:52.281727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1404/2808] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:52.287087 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.297379 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf1eac-f56e-4814-94cb-a3b8da53c915 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.302841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.303519 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.303756 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.304038 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.304697 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:38:52.305127 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.306723 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.319471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1405/2809] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:38:52.323896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.333974 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcbabd-4d47-42f7-a6be-c3a82415fc85 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.338700 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.339300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.339578 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.339885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.340524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.342421 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.460609 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1405/2810] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/projects => generated 355 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:38:52.464916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.475380 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01117df6-dc57-4d83-95f9-c219d6b8e00b admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.542680 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.543312 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.543601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.543912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.544559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.546533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.635982 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1406/2811] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/users => generated 278 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:38:52.640443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.651526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8e939-722d-48db-a81e-6c69d10fb647 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.656374 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.657042 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.657323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.657597 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.658252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.660606 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.678351 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1406/2812] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:38:52.682449 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.693501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14209f27-7b95-41e1-acbd-76bfc3fe7a0e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.700149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.700771 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.701049 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.701322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.702074 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.703912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dab69-047d-4e82-b219-33cdb84a08b3 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.721512 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1407/2813] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:38:52.726428 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.738758 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2feb-9652-4ecb-bd29-da3d1ef6a2e6 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:52.744129 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:52.744849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.745135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.745414 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] PATH_INFO: `/v3/projects/1eec6d0021c346a1b60ef580ac3a0fbe/users/1d14935fd76444eda4613d9e1ba9a873/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.761777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eec6d0021c346a1b60ef580ac3a0fbe, user_id=1d14935fd76444eda4613d9e1ba9a873, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:52.764546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:52.836727 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1407/2814] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:38:52 2022] PUT /identity/v3/projects/1eec6d0021c346a1b60ef580ac3a0fbe/users/1d14935fd76444eda4613d9e1ba9a873/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:52.849269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:52.849537 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:52.849815 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:52.879396 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] MFA Rules not processed for user `1d14935fd76444eda4613d9e1ba9a873`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:52.884868 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:52.950572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1408/2815] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:52.955175 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:52.976761 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:53.018023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a34a-11cf-4816-846a-71c6af19cc5f admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.023241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.023889 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.024173 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.024448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.025091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.026994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.174502 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1408/2816] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:38:52 2022] POST /identity/v3/projects => generated 355 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:38:53.179302 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.190833 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f86-2706-402e-9105-d0db38287065 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.205855 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:53.255707 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.258253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.258253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.258253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.258253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.259583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.350880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1409/2817] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:38:53 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:38:53.355085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.365814 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487b85-f9b7-41f7-89c0-35753c1eeadd admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.371004 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.371624 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.371934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.372209 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.372898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.374520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.386127 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1409/2818] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:38:53.391434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.402637 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69e716-767e-4550-b7ee-05f0c1d055c5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.407373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.408118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.408435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.408718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] PATH_INFO: `/v3/projects/ed579db8530d4a5e87963bc42f788736/users/ee3c10efcba24c129659cbb67f8d7cae/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.426887 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed579db8530d4a5e87963bc42f788736, user_id=ee3c10efcba24c129659cbb67f8d7cae, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.429789 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.504654 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1410/2819] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:38:53 2022] PUT /identity/v3/projects/ed579db8530d4a5e87963bc42f788736/users/ee3c10efcba24c129659cbb67f8d7cae/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:53.509444 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.521227 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a776e65-9100-41c9-9767-50f8fc7ee8f4 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.583916 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.584535 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.584849 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.585122 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.585796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.587472 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f24e97-aa08-4040-8797-9e3a0d7e85ca admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.599431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1410/2820] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:38:53.603791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.614901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe35fe2-c519-4a25-b4bd-d7ed7d320233 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.621058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.622014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.622433 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.622862 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] PATH_INFO: `/v3/projects/ed579db8530d4a5e87963bc42f788736/users/ee3c10efcba24c129659cbb67f8d7cae/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.625687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed579db8530d4a5e87963bc42f788736, user_id=ee3c10efcba24c129659cbb67f8d7cae, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.630640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.647901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1411/2821] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:38:53 2022] PUT /identity/v3/projects/ed579db8530d4a5e87963bc42f788736/users/ee3c10efcba24c129659cbb67f8d7cae/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:38:53.663763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.664202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.664623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.704257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] MFA Rules not processed for user `ee3c10efcba24c129659cbb67f8d7cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:38:53.705549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:53.781703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1411/2822] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:38:53.793559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:53.795317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:38:53.856906 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c9a0-d0b1-4029-b420-d6d6913f6dec admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:53.860567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:53.861180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:53.861460 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:53.861738 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:53.871753 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:53.873204 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:53.883682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1412/2823] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:38:54.137058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:38:54.147215 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e966a-7491-461b-b768-0f40010a6a86 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:38:54.151162 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:38:54.151823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:38:54.152112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:38:54.152392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:38:54.165946 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:38:54.217632 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:38:54.219151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f01c-d979-465f-ae43-a06c511a6dbf service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:38:54.229202 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1412/2824] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:03.879243 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:04.280999 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=21 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:12.073113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:12.139391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cd57-7ec6-4989-b428-36d35ce2dd6b service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:12.144712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:12.145335 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:12.145617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:12.145893 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] PATH_INFO: `/v3/users/1d14935fd76444eda4613d9e1ba9a873` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:12.147039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d14935fd76444eda4613d9e1ba9a873)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:12.148985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:12.330852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d14935fd76444eda4613d9e1ba9a873'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:12.406297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d14935fd76444eda4613d9e1ba9a873'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:12.406929 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d14935fd76444eda4613d9e1ba9a873'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:12.407551 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d14935fd76444eda4613d9e1ba9a873'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:12.420421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d14935fd76444eda4613d9e1ba9a873'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:12.432709 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1413/2825] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:12 2022] DELETE /identity/v3/users/1d14935fd76444eda4613d9e1ba9a873 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:12.442825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:12.443554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:12.443693 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:12.463424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:12.464667 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:12.523228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1413/2826] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:12.533178 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:12.534610 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:12.594317 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10308-15a7-4415-a86d-a4027db2132f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:12.597916 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:12.598521 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:12.598797 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:12.599062 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:12.609418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:12.610985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:12.622617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1414/2827] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:13.053984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:13.055429 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:13.075241 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ac5d-6e05-477f-91f7-e99b6e126e45 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:13.088806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:13.088806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:13.088806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:13.088806 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] PATH_INFO: `/v3/projects/1eec6d0021c346a1b60ef580ac3a0fbe` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:13.089214 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eec6d0021c346a1b60ef580ac3a0fbe)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:13.090260 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:13.190939 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eec6d0021c346a1b60ef580ac3a0fbe', 'request_id': 'req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.197754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eec6d0021c346a1b60ef580ac3a0fbe', 'request_id': 'req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.257252 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eec6d0021c346a1b60ef580ac3a0fbe', 'request_id': 'req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.269352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] The token cache is being invalidate because project 1eec6d0021c346a1b60ef580ac3a0fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:13.269673 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.271893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.272006 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1414/2828] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:13 2022] DELETE /identity/v3/projects/1eec6d0021c346a1b60ef580ac3a0fbe => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:13.276634 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:13.286277 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661fcce-3006-4b92-a410-326d06d0b769 service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:13.307412 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:13.343439 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:13.344113 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:13.344395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:13.344673 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] PATH_INFO: `/v3/users/ee3c10efcba24c129659cbb67f8d7cae` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:13.345852 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3c10efcba24c129659cbb67f8d7cae)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:13.348458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:13.547405 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3c10efcba24c129659cbb67f8d7cae'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.630096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3c10efcba24c129659cbb67f8d7cae'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.630826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3c10efcba24c129659cbb67f8d7cae'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.631452 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3c10efcba24c129659cbb67f8d7cae'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.648058 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-d7e40213-e410-4783-9847-44f46c161310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3c10efcba24c129659cbb67f8d7cae'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:13.662358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1415/2829] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:13 2022] DELETE /identity/v3/users/ee3c10efcba24c129659cbb67f8d7cae => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:14.076669 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.078253 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:14.096622 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae3c8-4b35-4fc1-8340-d1288feb1a7a admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.159517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.160178 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.160460 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.160737 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] PATH_INFO: `/v3/projects/ed579db8530d4a5e87963bc42f788736` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.161862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed579db8530d4a5e87963bc42f788736)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.163816 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.259019 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:14.275558 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed579db8530d4a5e87963bc42f788736', 'request_id': 'req-03b380de-d8d5-4e4a-a109-03cefc8e93b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:14.281798 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed579db8530d4a5e87963bc42f788736', 'request_id': 'req-03b380de-d8d5-4e4a-a109-03cefc8e93b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:14.360486 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed579db8530d4a5e87963bc42f788736', 'request_id': 'req-03b380de-d8d5-4e4a-a109-03cefc8e93b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:14.372892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] The token cache is being invalidate because project ed579db8530d4a5e87963bc42f788736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:14.373180 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:14.373815 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-03b380de-d8d5-4e4a-a109-03cefc8e93b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:14.375703 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1415/2830] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:14 2022] DELETE /identity/v3/projects/ed579db8530d4a5e87963bc42f788736 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:14.380872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.381373 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.381471 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.401189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:14.402524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:14.464124 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1416/2831] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:14.526712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.526997 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.527275 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.546977 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:14.548245 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:14.556898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1416/2832] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:14.561518 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.572184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac5900-7c31-4672-9b10-47f35bdc1392 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.577059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.577706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.578002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.578281 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.578932 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:39:14.579249 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.580995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.593338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1417/2833] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:39:14.597587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.608933 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c9c5d-e150-4ab8-b01d-00712a485abe None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.615932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.615932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.615932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.615932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.616575 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.618498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.711944 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1417/2834] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:14 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:39:14.716570 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.727088 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05571998-ced3-45d3-8db7-4aa20caab7de admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.753802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:14.792711 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.799349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.799349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.799349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.799349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.799349 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.876238 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1418/2835] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:14 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:39:14.882038 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.896517 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f9e2-b249-4b94-a759-103dbae176d9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.901288 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.902002 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.902276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.902551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.903190 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.904978 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.919871 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1418/2836] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:14.925142 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.936841 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7a108-e2bd-4ac5-a4c2-a28d8b370308 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.944108 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.944739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.945028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.945310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.945992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.947602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2faea-da36-45bd-aa0e-3e60257484de admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:14.958883 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1419/2837] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:14.963161 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:14.973427 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475c45-2b58-4a5e-bc22-38073b98390b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:14.978305 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:14.978961 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:14.979242 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:14.979520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] PATH_INFO: `/v3/projects/b062b58426d448a895425d8f57c1bf6d/users/7b735f0719ac4e9599d6f35963be4157/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:14.995553 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b062b58426d448a895425d8f57c1bf6d, user_id=7b735f0719ac4e9599d6f35963be4157, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:14.998285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.087103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1419/2838] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:14 2022] PUT /identity/v3/projects/b062b58426d448a895425d8f57c1bf6d/users/7b735f0719ac4e9599d6f35963be4157/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:15.097591 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.097879 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.098157 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.129931 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] MFA Rules not processed for user `7b735f0719ac4e9599d6f35963be4157`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:15.131060 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:15.196367 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1420/2839] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:39:15.200898 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.266390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4d60-eddb-4e6e-a6be-31f66db7eab9 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.274068 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.274956 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.275238 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.275512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.276170 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.278085 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.402519 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1420/2840] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:15 2022] POST /identity/v3/projects => generated 393 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:15.406533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.417984 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92690405-39ec-4f68-9ec9-4b9f14cafe7c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.485496 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.486115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.486406 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.486682 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.487322 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.490395 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.572097 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1421/2841] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:15 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:15.576118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.586243 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e53e2-6a28-49ef-95b2-31888342d75c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.590932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.591529 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.591842 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.592119 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.592755 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.594400 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.606037 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1421/2842] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:15.610876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.621555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799580f-ab4a-4cd9-aae5-91cf7d612dbb admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.626338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.626987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.627270 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.627545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] PATH_INFO: `/v3/projects/a2811663ed0542e69eded293e2abdd68/users/9badfb56ad794b34af12074d153bd45c/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.643507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2811663ed0542e69eded293e2abdd68, user_id=9badfb56ad794b34af12074d153bd45c, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.646224 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.714926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1422/2843] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:15 2022] PUT /identity/v3/projects/a2811663ed0542e69eded293e2abdd68/users/9badfb56ad794b34af12074d153bd45c/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:15.718823 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.729748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845265a-d46c-41b6-800d-6259b88932e0 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.749496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:15.787303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.787945 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.788233 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.788512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.789157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.790752 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60199e32-a9af-4f8a-a62c-b30535e00859 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.801016 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1422/2844] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:15.804937 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.810614 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:15.816039 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0899-e75f-4cfb-85e1-7bcfd54a6d1e admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:15.821138 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:15.821911 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.822196 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.822486 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] PATH_INFO: `/v3/projects/a2811663ed0542e69eded293e2abdd68/users/9badfb56ad794b34af12074d153bd45c/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.824743 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2811663ed0542e69eded293e2abdd68, user_id=9badfb56ad794b34af12074d153bd45c, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:15.829419 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:15.838497 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1423/2845] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:15 2022] PUT /identity/v3/projects/a2811663ed0542e69eded293e2abdd68/users/9badfb56ad794b34af12074d153bd45c/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:15.849917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:15.850208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:15.852364 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:15.878629 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] MFA Rules not processed for user `9badfb56ad794b34af12074d153bd45c`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:15.879924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:15.949526 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1423/2846] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:15.960795 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:15.962647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:16.023458 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b2b7-d703-4360-a6a7-3789f4ce2e37 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:16.027400 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:16.028033 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:16.028313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:16.028584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:16.038498 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:16.039933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:16.049459 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1424/2847] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:15 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:16.096347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:16.100872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:16.110498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51febad4-00c6-4d80-a5d2-f0dede8dc38b None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:16.114135 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:16.114738 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:16.115024 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:16.115304 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:16.172196 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:16.173637 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e88a09-6032-4d65-bd26-f34295c19a3e service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:16.182550 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1424/2848] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:20.794041 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:20.861545 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3216e-321b-437c-a254-0003dcc6a392 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:20.866967 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:20.867706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:20.867995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:20.868205 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] PATH_INFO: `/v3/users/7b735f0719ac4e9599d6f35963be4157` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:20.869372 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b735f0719ac4e9599d6f35963be4157)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:20.871360 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:21.021133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b735f0719ac4e9599d6f35963be4157'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.103527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b735f0719ac4e9599d6f35963be4157'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.104171 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b735f0719ac4e9599d6f35963be4157'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.104816 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b735f0719ac4e9599d6f35963be4157'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.118660 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b735f0719ac4e9599d6f35963be4157'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.133285 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1425/2849] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:20 2022] DELETE /identity/v3/users/7b735f0719ac4e9599d6f35963be4157 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:21.138631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:21.139136 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:21.139257 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:21.166879 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:21.169686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:21.212520 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=100 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:21.233439 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1425/2850] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:39:21.243409 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:21.245030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:21.258963 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:21.310463 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a518c2-3342-4175-bc21-0b423a0e0674 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:21.314391 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:21.314992 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:21.315269 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:21.315542 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:21.327599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:21.329075 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:21.339553 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1426/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:21.734051 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:21.735620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:21.751625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a49dc-acc5-413b-a2b5-875d83097f96 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:21.757980 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:21.758589 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:21.758872 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:21.759303 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] PATH_INFO: `/v3/projects/b062b58426d448a895425d8f57c1bf6d` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:21.760470 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b062b58426d448a895425d8f57c1bf6d)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:21.762394 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:21.876902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b062b58426d448a895425d8f57c1bf6d', 'request_id': 'req-1895d2d7-b76b-471d-a9b6-7ed134ffe822'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.883919 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b062b58426d448a895425d8f57c1bf6d', 'request_id': 'req-1895d2d7-b76b-471d-a9b6-7ed134ffe822'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.949313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b062b58426d448a895425d8f57c1bf6d', 'request_id': 'req-1895d2d7-b76b-471d-a9b6-7ed134ffe822'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.961548 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] The token cache is being invalidate because project b062b58426d448a895425d8f57c1bf6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:21.963873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.963873 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:21.964325 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1426/2852] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:21 2022] DELETE /identity/v3/projects/b062b58426d448a895425d8f57c1bf6d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:21.968377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:21.978323 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675193-7c57-42df-a8c2-191ba72bd64b service neutron] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:22.043095 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:22.043989 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:22.044554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:22.044554 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] PATH_INFO: `/v3/users/9badfb56ad794b34af12074d153bd45c` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:22.046002 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9badfb56ad794b34af12074d153bd45c)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:22.047588 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:22.301574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9badfb56ad794b34af12074d153bd45c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:22.406133 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9badfb56ad794b34af12074d153bd45c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:22.406826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9badfb56ad794b34af12074d153bd45c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:22.407533 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9badfb56ad794b34af12074d153bd45c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:22.422099 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a03f39d7-fcd8-4cc3-8bad-8e2db7b8faa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9badfb56ad794b34af12074d153bd45c'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:22.434901 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1427/2853] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:21 2022] DELETE /identity/v3/users/9badfb56ad794b34af12074d153bd45c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:22.847853 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:22.849803 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:22.866126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895d2d7-b76b-471d-a9b6-7ed134ffe822 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:22.925840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:22.927154 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:22.934418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:22.934418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] PATH_INFO: `/v3/projects/a2811663ed0542e69eded293e2abdd68` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:22.934418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2811663ed0542e69eded293e2abdd68)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:22.934418 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.017763 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2811663ed0542e69eded293e2abdd68', 'request_id': 'req-c4897693-c947-4803-a620-a452f2d6a562'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:23.024226 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2811663ed0542e69eded293e2abdd68', 'request_id': 'req-c4897693-c947-4803-a620-a452f2d6a562'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:23.102991 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2811663ed0542e69eded293e2abdd68', 'request_id': 'req-c4897693-c947-4803-a620-a452f2d6a562'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:23.112330 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:23.116603 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] The token cache is being invalidate because project a2811663ed0542e69eded293e2abdd68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81558) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:23.116835 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:23.117545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-c4897693-c947-4803-a620-a452f2d6a562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:23.119058 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1427/2854] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:22 2022] DELETE /identity/v3/projects/a2811663ed0542e69eded293e2abdd68 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:23.124052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744824d-0946-443e-9573-e85589fafd16 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.124550 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744824d-0946-443e-9573-e85589fafd16 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.124647 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744824d-0946-443e-9573-e85589fafd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.146001 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-d744824d-0946-443e-9573-e85589fafd16 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:23.147208 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-d744824d-0946-443e-9573-e85589fafd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:23.177319 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:23.208718 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1428/2855] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:23.216830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.221126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.221126 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.242137 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:23.243376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:23.255932 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1428/2856] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:23.260427 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744824d-0946-443e-9573-e85589fafd16 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.271858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744824d-0946-443e-9573-e85589fafd16 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.276583 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.280019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.280019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.280019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.280019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81559) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:39:23.280019 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.280702 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.298030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1429/2857] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:39:23.302984 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.312605 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b6f6d-784a-46a0-8688-776ea49c829c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.317309 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.317911 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.318448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.318716 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.319337 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.321282 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.397587 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1429/2858] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:23 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:23.401712 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.412569 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc62ac7-1875-4f55-839f-9e9b0e05efb5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.472909 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.474231 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.474514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.474787 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.475435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.477754 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.595628 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1430/2859] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:23 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 19:39:23.602240 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.613500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f70f7-7b0e-43c6-a505-5d0301cc6066 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.618728 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.619376 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.619684 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.619965 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.620623 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.622352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.633618 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1430/2860] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:39:23.640843 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.648729 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161d45d-5bfd-40b5-838d-2b76c86b805d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.653615 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.654228 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.654514 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.654792 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.655437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.657101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab50a3-7d02-4158-aa92-32cf569c1f7a admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.668287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1431/2861] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:23.676825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.687401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44306c6-5515-4e77-bb74-b2711087a3af admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.693500 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.694159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.694435 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.694708 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] PATH_INFO: `/v3/projects/c52a9d780a5245bcb709cf8a49a0aabe/users/9097f4940d1e45dc92d25f9ee18e2d7a/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.710764 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52a9d780a5245bcb709cf8a49a0aabe, user_id=9097f4940d1e45dc92d25f9ee18e2d7a, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:23.713713 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:23.812951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1431/2862] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:23 2022] PUT /identity/v3/projects/c52a9d780a5245bcb709cf8a49a0aabe/users/9097f4940d1e45dc92d25f9ee18e2d7a/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:23.819090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4dd38-60a5-4a8d-b7ae-b65aeb625402 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.822994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4dd38-60a5-4a8d-b7ae-b65aeb625402 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.822994 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4dd38-60a5-4a8d-b7ae-b65aeb625402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.841995 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-84a4dd38-60a5-4a8d-b7ae-b65aeb625402 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:23.843221 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-84a4dd38-60a5-4a8d-b7ae-b65aeb625402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:23.911567 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1432/2863] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:23.921652 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:23.923748 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:23.983251 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80c723-5963-44d1-bd6a-2192bb64bd41 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:23.987118 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:23.987917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:23.988229 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:23.988505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:23.998880 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:24.000390 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:24.010496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1432/2864] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:25.315927 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.316470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.316470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.353895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] MFA Rules not processed for user `9097f4940d1e45dc92d25f9ee18e2d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:25.355120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:25.385701 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:25.425024 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1433/2865] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:25.429694 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.440081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed5e45-0bcf-4094-aed8-90f24d4bf4d7 service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.446285 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.446864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.447151 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.447426 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.448113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.450736 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.534658 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1433/2866] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:25 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:25.539207 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.550125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27604f7-80ed-47c3-855b-af1f07e2c3b6 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.607704 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.608318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.608602 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.608878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.609527 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.611417 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.691543 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1434/2867] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:25 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:25.695917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.706532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcc1ec-4272-4d7a-82c6-2115e7742034 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.711664 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.712272 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.712554 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.712840 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.713477 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.715078 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.726532 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1434/2868] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 19:39:25.732447 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.742599 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfabe5-2699-44ef-84d0-5887ed515e03 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.747444 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.748134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.748418 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.748693 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] PATH_INFO: `/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d/users/997cced9bda643f9927997b197b7f41d/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.763347 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd837ad2cf8845f49f7a11d3c0c7538d, user_id=997cced9bda643f9927997b197b7f41d, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.766008 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.846434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1435/2869] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:25 2022] PUT /identity/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d/users/997cced9bda643f9927997b197b7f41d/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:25.850343 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.860220 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde0b5-a27f-43eb-a15e-1b9a41e22a41 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.918659 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.919410 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.919925 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.920311 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.921069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.922953 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897349-6345-4153-9939-99241dd39414 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.928912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:25.933886 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1435/2870] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:25.938358 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:25.949747 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf667-7fc2-4c48-99b5-6a6e7a4bc5bb admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:25.954507 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:25.955153 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:25.955436 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:25.955739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] PATH_INFO: `/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d/users/997cced9bda643f9927997b197b7f41d/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:25.957737 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd837ad2cf8845f49f7a11d3c0c7538d, user_id=997cced9bda643f9927997b197b7f41d, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:25.960973 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:25.971910 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1436/2871] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:25 2022] PUT /identity/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d/users/997cced9bda643f9927997b197b7f41d/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:27.300534 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:27.301157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:27.301327 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:27.333014 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] MFA Rules not processed for user `997cced9bda643f9927997b197b7f41d`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:27.336280 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:27.413064 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1436/2872] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:27.424778 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:27.426524 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:27.486502 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879b4c-fecb-4b6c-8c0a-37e255b6bdfa admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:27.490230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:27.491332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:27.491616 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:27.491934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:27.524934 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:27.547608 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:27.548993 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:27.558410 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1437/2873] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:29.251890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:29.253287 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:29.266295 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4063013-82cc-4e00-8416-6cadac02879e None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:29.269891 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:29.270504 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:29.270783 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:29.271055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:29.280620 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:29.286604 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:29.292731 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1437/2874] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:37.600255 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=11 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:39.288402 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=40 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:44.912092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:44.913605 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:45.002996 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac652-e2c6-4398-96fd-03b4d44c2365 service cinder] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:45.008120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:45.008739 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:45.009030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:45.010053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:45.011625 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:45.024191 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:45.027983 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:45.037041 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:45.040456 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1438/2875] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:45.108827 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9f97-6053-4536-a32b-dde58af7b183 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:45.113654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:45.114308 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:45.114591 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:45.114868 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] PATH_INFO: `/v3/users/9097f4940d1e45dc92d25f9ee18e2d7a` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:45.116187 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9097f4940d1e45dc92d25f9ee18e2d7a)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:45.120128 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:45.295131 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9097f4940d1e45dc92d25f9ee18e2d7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.365825 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9097f4940d1e45dc92d25f9ee18e2d7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.366453 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9097f4940d1e45dc92d25f9ee18e2d7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.367069 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9097f4940d1e45dc92d25f9ee18e2d7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.375445 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9097f4940d1e45dc92d25f9ee18e2d7a'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.393387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1438/2876] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:45 2022] DELETE /identity/v3/users/9097f4940d1e45dc92d25f9ee18e2d7a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 19:39:45.659986 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:45.673147 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961e17-69ff-4b47-84f9-680b017fa53e service placement] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:45.731437 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:45.732092 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:45.732375 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:45.732650 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] PATH_INFO: `/v3/projects/c52a9d780a5245bcb709cf8a49a0aabe` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:45.733766 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52a9d780a5245bcb709cf8a49a0aabe)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:45.735717 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:45.869354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52a9d780a5245bcb709cf8a49a0aabe', 'request_id': 'req-b1b750b8-535e-440a-9a3a-acab374f71ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.875559 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52a9d780a5245bcb709cf8a49a0aabe', 'request_id': 'req-b1b750b8-535e-440a-9a3a-acab374f71ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.957940 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52a9d780a5245bcb709cf8a49a0aabe', 'request_id': 'req-b1b750b8-535e-440a-9a3a-acab374f71ad'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.969829 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] The token cache is being invalidate because project c52a9d780a5245bcb709cf8a49a0aabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:45.970125 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.970774 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:45.972313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1439/2877] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:45 2022] DELETE /identity/v3/projects/c52a9d780a5245bcb709cf8a49a0aabe => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:45.976387 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:45.977884 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:45.990970 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26adce-454b-43d0-91bf-521190cea1e6 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:46.047124 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:46.047870 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:46.048049 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:46.048359 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] PATH_INFO: `/v3/users/997cced9bda643f9927997b197b7f41d` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:46.049466 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=997cced9bda643f9927997b197b7f41d)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:46.051377 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:46.073199 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:46.272281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997cced9bda643f9927997b197b7f41d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.331448 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997cced9bda643f9927997b197b7f41d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.332139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997cced9bda643f9927997b197b7f41d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.332732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997cced9bda643f9927997b197b7f41d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.340903 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-6e33bc63-03f9-4dd8-af9c-511feff28b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997cced9bda643f9927997b197b7f41d'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.360268 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1439/2878] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:45 2022] DELETE /identity/v3/users/997cced9bda643f9927997b197b7f41d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:46.590440 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:46.591912 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:46.606557 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b750b8-535e-440a-9a3a-acab374f71ad admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:46.626155 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:46.675362 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:46.676048 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:46.676337 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:46.676617 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] PATH_INFO: `/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:46.677772 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd837ad2cf8845f49f7a11d3c0c7538d)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:46.679759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:46.784287 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd837ad2cf8845f49f7a11d3c0c7538d', 'request_id': 'req-3cbb3c3d-fcab-416a-a497-a840bb3813e7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.790975 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd837ad2cf8845f49f7a11d3c0c7538d', 'request_id': 'req-3cbb3c3d-fcab-416a-a497-a840bb3813e7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.851555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd837ad2cf8845f49f7a11d3c0c7538d', 'request_id': 'req-3cbb3c3d-fcab-416a-a497-a840bb3813e7'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.863455 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] The token cache is being invalidate because project bd837ad2cf8845f49f7a11d3c0c7538d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:46.863802 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.864478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-3cbb3c3d-fcab-416a-a497-a840bb3813e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:46.865969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1440/2879] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:46 2022] DELETE /identity/v3/projects/bd837ad2cf8845f49f7a11d3c0c7538d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:46.871091 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:46.871373 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:46.872350 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:46.891826 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:46.893157 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:46.958082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1440/2880] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:46.966622 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:46.967165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:46.967165 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:46.993812 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:46.995052 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:47.003096 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1441/2881] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:47.008714 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.018924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f798ba7-7612-426f-b888-e8d4d6307af8 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.023631 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.024259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.024539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.024811 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.025456 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:39:47.025757 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.027360 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.042095 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1441/2882] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 19:39:47.047416 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.059598 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb648b1-6006-4994-bf4c-5dcd1f46e34c None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.066470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.067093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.067377 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.067671 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.068690 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.070501 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.150341 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1442/2883] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:47 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:47.155163 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.165903 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6922-dc60-4d1c-a132-063e6df7f0e1 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.223796 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.224541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.224832 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.225112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.225744 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.227747 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.310256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1442/2884] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:47 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:47.314710 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.325483 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab9201-a2aa-4390-a9ac-f3553101bac5 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.330278 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.330876 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.337206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.337206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.337206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.337206 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.345150 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1443/2885] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:47.349273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.359976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec96c7-1cfb-432d-a407-abc99ed9e4f7 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.366012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.366654 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.366934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.367208 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.367885 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.369563 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3dda-6de9-4a6d-8b18-cf6f039aa7c2 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.381222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1443/2886] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:47.390640 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.407442 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a1cd-64b3-4d5b-9b65-a7887cb2d3f9 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.413779 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.415811 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.416803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.417090 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] PATH_INFO: `/v3/projects/e6ac8cadc3364dce87b2e7d57014258e/users/4ad1833af0744af095d2ef142cbe5b8e/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.433363 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ac8cadc3364dce87b2e7d57014258e, user_id=4ad1833af0744af095d2ef142cbe5b8e, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.436070 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.578677 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1444/2887] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:47 2022] PUT /identity/v3/projects/e6ac8cadc3364dce87b2e7d57014258e/users/4ad1833af0744af095d2ef142cbe5b8e/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:47.594934 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.617892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.617892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.617892 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:47.627349 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] MFA Rules not processed for user `4ad1833af0744af095d2ef142cbe5b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:47.629143 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:47.705030 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1444/2888] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 19:39:47.709576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.770098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:47.785540 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8050-35c9-4236-8b4a-cca4793fe355 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.790666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.795581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.795581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.795581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.795581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.795581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:47.864618 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1445/2889] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:47 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:47.868938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:47.880572 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8e33d-7fd4-47a4-8ddc-7e688e24782c None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:47.941461 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:47.942077 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:47.942363 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:47.942641 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:47.943281 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:47.945549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.040546 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1445/2890] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:47 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:48.045093 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.056966 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756d89-208e-4a1f-9e63-d5227e495979 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.062584 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.063230 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.063492 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.063803 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.064453 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.066053 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.084907 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1446/2891] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:48.089358 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.099900 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63a33-064a-45e0-ac3e-6a3db00de631 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.105055 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.106754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.106754 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.107100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.107523 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.111412 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda472-9a75-4e3d-b74a-7fdb41f3f94e admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.131021 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1446/2892] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:48.135381 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.146083 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf16b3-af8b-43a6-9ffd-e99d298154c8 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.151118 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.154172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.154172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.154172 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] PATH_INFO: `/v3/projects/42dab64fad14467ba3de9befa85f1764/users/7712dbf48d9141e4a3104356093539a7/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.169229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42dab64fad14467ba3de9befa85f1764, user_id=7712dbf48d9141e4a3104356093539a7, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.172201 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.280880 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1447/2893] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:48 2022] PUT /identity/v3/projects/42dab64fad14467ba3de9befa85f1764/users/7712dbf48d9141e4a3104356093539a7/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:48.291635 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.292216 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.292323 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.329496 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] MFA Rules not processed for user `7712dbf48d9141e4a3104356093539a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:48.330732 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:48.400464 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1447/2894] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:48.405535 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.469960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d021-50d5-4f6b-88f1-44ff0f1d5a18 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.475487 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.476390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.476770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.477139 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.478046 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.480282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.596184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1448/2895] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:39:48 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:48.600023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.605655 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:48.611938 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366192b-3abb-423a-a826-0cb4a86386dd None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.666574 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.667113 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.667458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.667688 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.668928 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.671431 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.760685 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1448/2896] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:39:48 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:39:48.768981 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.780585 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988b9f-77bd-4d15-b5a0-68da1fcc054f admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.785203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.785798 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.786084 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.786473 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.787114 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.788742 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.799950 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1449/2897] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:48.804276 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.817125 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd80f1d-590a-422c-84c6-e24296b1782b admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.825228 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.825878 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.826543 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.826859 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] PATH_INFO: `/v3/projects/7baa720be5194152bc239a3f082572eb/users/df84f95c18534c6bbfa9b494e55e15b0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.842917 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7baa720be5194152bc239a3f082572eb, user_id=df84f95c18534c6bbfa9b494e55e15b0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.845579 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:48.920986 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1449/2898] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:48 2022] PUT /identity/v3/projects/7baa720be5194152bc239a3f082572eb/users/df84f95c18534c6bbfa9b494e55e15b0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:48.924947 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:48.935354 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda396b-4230-4e3b-8155-ce7cd4e79156 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:48.992964 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:48.993576 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:48.993866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:48.994149 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:48.994791 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:48.996426 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c0492-97de-40a5-96d9-5b3c475b7764 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:49.006506 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1450/2899] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:39:49.010724 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:49.021380 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1542a2-761e-44af-8da2-4207138d41d3 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:49.026393 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:49.027033 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:49.027313 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:49.027625 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] PATH_INFO: `/v3/projects/7baa720be5194152bc239a3f082572eb/users/df84f95c18534c6bbfa9b494e55e15b0/roles/1ae7138219844425af3d0d3f1ef8ce39` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:49.030076 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7baa720be5194152bc239a3f082572eb, user_id=df84f95c18534c6bbfa9b494e55e15b0, role_id=1ae7138219844425af3d0d3f1ef8ce39)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:49.032745 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:49.043057 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1450/2900] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:39:49 2022] PUT /identity/v3/projects/7baa720be5194152bc239a3f082572eb/users/df84f95c18534c6bbfa9b494e55e15b0/roles/1ae7138219844425af3d0d3f1ef8ce39 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:49.053679 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:49.054274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:49.054274 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:49.068758 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:49.084180 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] MFA Rules not processed for user `df84f95c18534c6bbfa9b494e55e15b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:49.087703 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:49.178920 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1451/2901] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:49.190718 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:49.192551 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:49.263793 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e235fb-35c6-4f26-b9ef-0e15ee58e8cc admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:49.269993 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:49.271023 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:49.271552 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:49.272108 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:49.338896 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:49.340836 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:49.349951 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1451/2902] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:50.788609 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:50.790156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:50.869329 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1026a5e-6065-4bca-95cc-e2970fae7208 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:50.874985 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:50.875596 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:50.875913 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:50.876186 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:50.877953 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:50.891607 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:50.893332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:50.904633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1452/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:51.412489 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:51.417715 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:51.426213 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70bec6-3b0f-47a1-8459-b06d6929273d service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:51.430277 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:51.430902 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:51.431188 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:51.431465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:51.437959 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:51.504613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:51.507070 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:51.519073 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1452/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:52.970826 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:52.985633 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a8973-2e84-4be4-97e3-87a984d9485c service placement] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:52.989266 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:52.989871 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:52.990156 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:52.990441 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:53.000263 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:53.001695 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f33f1-3b8a-4849-93ac-88f97b70bd17 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:53.011489 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1453/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 19:39:57.633626 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:57.697352 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a19a4-6a34-414a-8311-55d442cae551 service cinder] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:57.702686 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:57.703319 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:57.703848 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:57.704001 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] PATH_INFO: `/v3/users/4ad1833af0744af095d2ef142cbe5b8e` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:57.705114 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad1833af0744af095d2ef142cbe5b8e)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:57.707082 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:57.877482 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad1833af0744af095d2ef142cbe5b8e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:57.954273 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad1833af0744af095d2ef142cbe5b8e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:57.954901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad1833af0744af095d2ef142cbe5b8e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:57.955505 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad1833af0744af095d2ef142cbe5b8e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:57.963256 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad1833af0744af095d2ef142cbe5b8e'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:57.981709 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1453/2906] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:57 2022] DELETE /identity/v3/users/4ad1833af0744af095d2ef142cbe5b8e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:57.986403 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:57.986892 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:57.987005 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:58.007131 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:39:58.008369 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:58.070863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1454/2907] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:39:58.081378 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:58.082830 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:58.114817 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:58.142935 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f547e26-d32d-482b-89f2-07dafebd940d admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:58.146469 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:58.147103 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:58.147379 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:58.147671 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:58.157112 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:58.158544 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:58.168750 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1454/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:39:58.533561 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:58.535203 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:58.550013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8a3b6-f225-46e3-a121-e4c0ef175162 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:58.552926 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:58.553572 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:58.553856 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:58.554297 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] PATH_INFO: `/v3/projects/e6ac8cadc3364dce87b2e7d57014258e` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:58.555954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ac8cadc3364dce87b2e7d57014258e)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:58.557957 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:58.560478 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=100 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:58.664845 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ac8cadc3364dce87b2e7d57014258e', 'request_id': 'req-dd64104a-b448-4ff4-af15-31dd1a30d378'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:58.671137 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ac8cadc3364dce87b2e7d57014258e', 'request_id': 'req-dd64104a-b448-4ff4-af15-31dd1a30d378'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:58.771767 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ac8cadc3364dce87b2e7d57014258e', 'request_id': 'req-dd64104a-b448-4ff4-af15-31dd1a30d378'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:58.784010 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] The token cache is being invalidate because project e6ac8cadc3364dce87b2e7d57014258e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:58.784313 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:58.784969 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:58.786474 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1455/2909] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:58 2022] DELETE /identity/v3/projects/e6ac8cadc3364dce87b2e7d57014258e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:58.790599 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:58.800512 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835531e-86b0-465c-93b0-ff6ac41c3d0f service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:58.859810 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:58.860443 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:58.861152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:58.861152 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] PATH_INFO: `/v3/users/7712dbf48d9141e4a3104356093539a7` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:58.862279 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7712dbf48d9141e4a3104356093539a7)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:58.864222 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:59.082424 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7712dbf48d9141e4a3104356093539a7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.154864 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7712dbf48d9141e4a3104356093539a7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.155525 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7712dbf48d9141e4a3104356093539a7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.156172 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7712dbf48d9141e4a3104356093539a7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.164577 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7712dbf48d9141e4a3104356093539a7'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.184012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1455/2910] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:58 2022] DELETE /identity/v3/users/7712dbf48d9141e4a3104356093539a7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:59.555056 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:59.556549 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:59.570434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64104a-b448-4ff4-af15-31dd1a30d378 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:59.628282 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:59.628914 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:59.629305 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:59.629581 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] PATH_INFO: `/v3/projects/42dab64fad14467ba3de9befa85f1764` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:59.630687 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42dab64fad14467ba3de9befa85f1764)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:59.632870 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:39:59.732326 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42dab64fad14467ba3de9befa85f1764', 'request_id': 'req-a0a063d9-d86c-4972-bb8d-82d16a24b749'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.738531 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42dab64fad14467ba3de9befa85f1764', 'request_id': 'req-a0a063d9-d86c-4972-bb8d-82d16a24b749'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.782338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42dab64fad14467ba3de9befa85f1764', 'request_id': 'req-a0a063d9-d86c-4972-bb8d-82d16a24b749'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.793366 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] The token cache is being invalidate because project 42dab64fad14467ba3de9befa85f1764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:39:59.793684 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.794318 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:39:59.795924 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1456/2911] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:39:59 2022] DELETE /identity/v3/projects/42dab64fad14467ba3de9befa85f1764 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:39:59.800026 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:39:59.801498 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:39:59.815009 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78658c6e-2037-4ab3-83a9-a16d77ec8daf admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:39:59.862090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:39:59.874774 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:39:59.875401 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:39:59.875712 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:39:59.875990 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] PATH_INFO: `/v3/users/df84f95c18534c6bbfa9b494e55e15b0` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:39:59.877138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df84f95c18534c6bbfa9b494e55e15b0)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:39:59.879107 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:00.038090 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df84f95c18534c6bbfa9b494e55e15b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.102642 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df84f95c18534c6bbfa9b494e55e15b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.103262 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df84f95c18534c6bbfa9b494e55e15b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.103897 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df84f95c18534c6bbfa9b494e55e15b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.112139 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-62ab2c60-9e1d-40d6-ad33-13df6d93b562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df84f95c18534c6bbfa9b494e55e15b0'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.130937 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1456/2912] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:39:59 2022] DELETE /identity/v3/users/df84f95c18534c6bbfa9b494e55e15b0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:40:00.533878 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:00.535434 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:00.551189 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a063d9-d86c-4972-bb8d-82d16a24b749 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:00.572177 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:00.610026 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:00.610574 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:00.610857 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:00.611134 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] PATH_INFO: `/v3/projects/7baa720be5194152bc239a3f082572eb` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:00.612332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7baa720be5194152bc239a3f082572eb)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:00.614222 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:00.726390 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7baa720be5194152bc239a3f082572eb', 'request_id': 'req-835af7d7-fece-4e0a-b58b-87999619879d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.733184 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7baa720be5194152bc239a3f082572eb', 'request_id': 'req-835af7d7-fece-4e0a-b58b-87999619879d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.805759 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7baa720be5194152bc239a3f082572eb', 'request_id': 'req-835af7d7-fece-4e0a-b58b-87999619879d'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.816639 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] The token cache is being invalidate because project 7baa720be5194152bc239a3f082572eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:40:00.816933 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.817565 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-835af7d7-fece-4e0a-b58b-87999619879d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:00.819310 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1457/2913] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:40:00 2022] DELETE /identity/v3/projects/7baa720be5194152bc239a3f082572eb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:40:00.824247 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:00.824777 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:00.824890 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:00.844682 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:40:00.845888 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:00.904302 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1457/2914] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:40:00.913385 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:00.913674 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:00.913954 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:00.933858 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:40:00.935068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:00.943030 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1458/2915] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:40:00.947602 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:00.957893 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d217f8-51ef-4a15-8919-4cd805c5c842 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:00.962541 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:00.964549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:00.964549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:00.964549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:00.964549 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81558) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 19:40:00.964924 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:00.966820 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:00.978458 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1458/2916] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 15 19:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 19:40:00.982823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:00.992853 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4963b1-16e9-4c45-8377-de329921bfc8 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:00.997271 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:00.997866 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:00.998190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:00.998470 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:00.999101 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.001014 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.064482 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1459/2917] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 15 19:40:00 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:40:01.069040 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:01.079159 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573a9d-bfe3-4333-951d-c4a5f09f8f98 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:01.129012 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:01.135722 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:01.136357 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.136638 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.136912 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] PATH_INFO: `/v3/users` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.137940 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.139976 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.223799 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1459/2918] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 15 19:40:01 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 19:40:01.228094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:01.239068 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287cf4b-b9a0-4317-9570-d067176d2dd2 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:01.243809 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:01.244423 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.244706 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.244987 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] PATH_INFO: `/v3/roles` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.245630 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.247214 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.257432 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1460/2919] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:40:01.264613 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:01.274985 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e24e0-f043-4bd2-9a73-bab64e3fec94 admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:01.280522 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:01.281138 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.281423 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.281697 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.282368 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.283994 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf03ac-89c8-4e2a-859d-f748a4b50857 admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.295405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1460/2920] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 15 19:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 19:40:01.299421 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:01.309120 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567b82f-4751-45fd-a766-f79438e7676d admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:01.313700 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:01.314332 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.314666 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.314891 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] PATH_INFO: `/v3/projects/815d376382fe47348cfa668bf5d17c76/users/052649c4080244ab8a6f401be32a7e33/roles/852ee84db43e4ddeba945be2f4d8f1ba` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.329601 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815d376382fe47348cfa668bf5d17c76, user_id=052649c4080244ab8a6f401be32a7e33, role_id=852ee84db43e4ddeba945be2f4d8f1ba)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.332265 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.394823 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1461/2921] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 15 19:40:01 2022] PUT /identity/v3/projects/815d376382fe47348cfa668bf5d17c76/users/052649c4080244ab8a6f401be32a7e33/roles/852ee84db43e4ddeba945be2f4d8f1ba => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:40:01.407177 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] REQUEST_METHOD: `POST` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.407450 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.407766 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.436300 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.auth.core [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] MFA Rules not processed for user `052649c4080244ab8a6f401be32a7e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81558) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:40:01.437509 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:01.501542 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1461/2922] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:40:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:40:01.512229 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:01.513915 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:01.571872 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:01.575162 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf56-0bb6-45ad-93f2-9cf60fdccd23 admin admin] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:01.578960 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de7b02bd5864be1b8e8bc6e06dddd55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:01.589863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] REQUEST_METHOD: `GET` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:01.589863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:01.589863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:01.589863 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:01.591694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d2a49-e109-426c-9392-07db24c87c67 service cinder] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:01.601013 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1462/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:40:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:40:11.498699 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:11.662136 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=4 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:12.954465 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:13.020949 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca747c-924a-49f3-ba67-06162e0b67a2 None None] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:13.026092 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:13.026756 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:13.027100 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:13.027372 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] PATH_INFO: `/v3/users/052649c4080244ab8a6f401be32a7e33` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:13.028545 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] RBAC: Authorizing `identity:delete_user(user_id=052649c4080244ab8a6f401be32a7e33)` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:13.034862 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:13.202730 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052649c4080244ab8a6f401be32a7e33'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:13.285607 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052649c4080244ab8a6f401be32a7e33'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:13.286249 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052649c4080244ab8a6f401be32a7e33'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:13.286854 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052649c4080244ab8a6f401be32a7e33'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:13.294787 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.notifications [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052649c4080244ab8a6f401be32a7e33'} {{(pid=81558) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:13.314839 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1462/2924] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 15 19:40:12 2022] DELETE /identity/v3/users/052649c4080244ab8a6f401be32a7e33 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:40:13.319694 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] REQUEST_METHOD: `POST` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:13.320252 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:13.320425 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:13.341059 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.auth.core [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] MFA Rules not processed for user `ddb604f3212045e99bbc5f99912931b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81559) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 19:40:13.346115 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:13.403085 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1463/2925] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 15 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 19:40:13.412901 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:13.414407 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:13.472681 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74b001-9c1d-451b-8bcb-f437d970055c admin admin] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:13.476224 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed6b2414b84f4ba0a59b3700b122ba66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:13.476833 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:13.477115 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:13.477392 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:13.487080 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:13.488539 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:13.498516 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1463/2926] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:40:13.894094 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] Authenticating user token {{(pid=81559) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:40:13.897435 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.fernet_utils [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81559) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:40:13.908770 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156b45e-1327-41af-87fd-1a549afebc02 None None] Validating token access rules against request {{(pid=81559) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:40:13.914450 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddb604f3212045e99bbc5f99912931b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6238bc86e5b040f1914e478da12747a8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81559) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:40:13.915253 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:40:13.915667 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:40:13.916040 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] PATH_INFO: `/v3/projects/815d376382fe47348cfa668bf5d17c76` {{(pid=81559) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:40:13.917338 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815d376382fe47348cfa668bf5d17c76)` {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:40:13.919365 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] RBAC: Authorization granted {{(pid=81559) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:40:14.030190 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815d376382fe47348cfa668bf5d17c76', 'request_id': 'req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:14.036555 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815d376382fe47348cfa668bf5d17c76', 'request_id': 'req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:14.126465 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815d376382fe47348cfa668bf5d17c76', 'request_id': 'req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9'} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:14.140917 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] The token cache is being invalidate because project 815d376382fe47348cfa668bf5d17c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81559) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 19:40:14.141210 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:14.141895 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG keystone.notifications [None req-f5dbefa1-66b0-4349-a5e8-88bd3635dec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81559) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 19:40:14.144098 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: [pid: 81559|app: 0|req: 1464/2927] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 15 19:40:13 2022] DELETE /identity/v3/projects/815d376382fe47348cfa668bf5d17c76 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 19:40:23.494081 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:40:24.136028 dv-cinder-849846-10-iscsi devstack@keystone.service[81559]: DEBUG dbcounter [-] [81559] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81559) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 19:44:53.392054 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] Authenticating user token {{(pid=81558) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 19:44:53.394259 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:44:53.531612 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7d97e-0c3f-49e0-be28-ac7f281b5c5c service neutron] Validating token access rules against request {{(pid=81558) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 19:44:53.591405 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc38e83dcf4b4cd794bbc5c550d7fc05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcb07dc6930943a98070c1ab5007d449', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81558) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 19:44:53.592052 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] REQUEST_METHOD: `GET` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 19:44:53.592338 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] SCRIPT_NAME: `/identity` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 19:44:53.592615 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81558) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 19:44:53.594347 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.fernet_utils [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81558) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 19:44:53.606485 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 19:44:53.609221 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01629694-59c6-457d-bc35-88d4a3f26323 service placement] RBAC: Authorization granted {{(pid=81558) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 19:44:53.618524 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: [pid: 81558|app: 0|req: 1464/2928] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 15 19:44:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 19:45:03.614217 dv-cinder-849846-10-iscsi devstack@keystone.service[81558]: DEBUG dbcounter [-] [81558] Writing DB stats keystone:SELECT=89 {{(pid=81558) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}